-- Logs begin at Tue 2022-06-21 18:11:26 UTC, end at Tue 2022-06-21 19:27:04 UTC. -- Jun 21 18:48:30.466446 dv-cinder-846303-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 21 18:48:30.476022 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 21 18:48:30 2022] *** Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: nodename: dv-cinder-846303-1-iscsi Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: machine: x86_64 Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: clock source: unix Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: pcre jit disabled Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: detected number of CPU cores: 4 Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: current working directory: / Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: detected binary path: /usr/bin/uwsgi-core Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: your processes number limit is 47680 Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: your memory page size is 4096 bytes Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: detected max file descriptor number: 2048 Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: lock engine: pthread robust mutexes Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: thunder lock: enabled Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 21 18:48:30.477369 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 21 18:48:30.521372 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: Python main interpreter initialized at 0x55fc6d35f9c0 Jun 21 18:48:30.521372 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: python threads support enabled Jun 21 18:48:30.521372 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: your server socket listen backlog is limited to 100 connections Jun 21 18:48:30.521372 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: your mercy for graceful operations on workers is 90 seconds Jun 21 18:48:30.521716 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: mapped 403077 bytes (393 KB) for 2 cores Jun 21 18:48:30.521790 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: *** Operational MODE: preforking *** Jun 21 18:48:30.522072 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: *** uWSGI is running in multiple interpreter mode *** Jun 21 18:48:30.522072 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: spawned uWSGI master process (pid: 80144) Jun 21 18:48:30.522273 dv-cinder-846303-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 21 18:48:30.526430 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: spawned uWSGI worker 1 (pid: 80157, cores: 1) Jun 21 18:48:30.526430 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: spawned uWSGI worker 2 (pid: 80158, cores: 1) Jun 21 18:48:30.526430 dv-cinder-846303-1-iscsi devstack@keystone.service[80144]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 21 18:48:35.957406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:48:35.957406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: from cryptography.utils import int_from_bytes Jun 21 18:48:35.958184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:48:35.958184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: from cryptography.utils import int_from_bytes Jun 21 18:48:35.958321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:48:35.958321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: from cryptography.utils import int_from_bytes Jun 21 18:48:35.958741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 18:48:35.958741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: from cryptography.utils import int_from_bytes Jun 21 18:48:36.006867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 18:48:36.007038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 18:48:36.007236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] command line args: [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 18:48:36.007505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 18:48:36.007583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 18:48:36.007687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ================================================================================ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 18:48:36.007776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 18:48:36.007941 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] command line args: [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 18:48:36.008025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] admin_token = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.008187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] config_dir = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.008264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 18:48:36.008377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.008467 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ================================================================================ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 18:48:36.008546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] config_source = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.008621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] admin_token = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.008728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] debug = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.008869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] config_dir = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.008956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.009237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.009338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] config_source = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.009444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.009547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] debug = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.009651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.009753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.009894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_config_append = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_dir = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.010944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_file = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_config_append = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_dir = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_file = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.011946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.012018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.012135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.012216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.012339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.012470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.012569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.012807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.013044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.013362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.013497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.013631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.013756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.013833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.013906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.013972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.014037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.014108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.014289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.014394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.014514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.014759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.014854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.014924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] publish_errors = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.015018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.015177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.015287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.015439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.015542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.015651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.015769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] publish_errors = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016283 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] standard_threads = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.016936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.017057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_journal = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.017148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.017220 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_json = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.017352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.017440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_stderr = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.017541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_syslog = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.017626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] standard_threads = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.017773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.017856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.018030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.018107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.018215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.018307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.018380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_journal = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.018516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.018594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_json = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.018731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_stderr = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.018813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.018913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_syslog = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.018992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.019147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 18:48:36.019225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.019409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.019487 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.019654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.019732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.019908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.019984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.020151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.020226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.020406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.020484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.020638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.020714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.020885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.020963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.021141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.021223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.021391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.021467 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.021641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.021720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.021877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.021961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.022119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.022195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.022368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.022446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.022595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.022670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.022835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.022912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.023088 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.023164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.023332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.023411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.023588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.023663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.023837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.023914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.024069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.024154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.024301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.024382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.024531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.024614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.024766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.024845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.025014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.025128 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.025289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.025379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.025520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.025603 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.025750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.025828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.025995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.026078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.026236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.026314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.026472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.026547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.026721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.026806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.026953 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.027029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.027187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.027263 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.027422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.027509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.027652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.027730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.027882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.027965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.028112 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.028268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.028349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.028420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.028565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.028648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.028814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.028900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.029046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.029162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.029308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.029384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.029537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.030111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.030292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.030369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.030535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.030614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.030783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.030869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.031014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.031128 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.031254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.031358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.031488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.031597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.031722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.031824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.031960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.032068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.032215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.032302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.032410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.032542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.032663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.032786 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.032897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.032996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.033158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.033244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.033391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.033479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.033635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.033747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.033872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.033969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.034102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.034204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.034342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.034441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.034582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.034677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.034823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.034914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.035075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.035174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.035318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.035412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.035553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.035632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.035788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.035870 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.036023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.036103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.036258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.036333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.036506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.036606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.036745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.036834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.036982 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.037059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.037373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.037475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.037676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.037754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.037825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.037902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.037981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.038054 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.038204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.038295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.038392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.038471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.038655 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.038737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.038897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.038976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.039136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.039213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.039375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.039452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.039615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.039695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.039855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.039932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.040089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.040166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.040316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.040397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.040550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.040629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.040782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.040860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.041012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.041125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.041272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.041359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.041503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.041582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.041741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.041820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.041975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.042062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.042210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.042288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.042446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.042524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.042684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.042762 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.042921 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.043000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.043166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.043245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.043394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.external = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.043473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.043623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.043701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.043874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.043959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.044134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.044218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.044365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.password = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.044443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.044596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.token = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.044707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.044835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.044924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.external = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.045070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.045169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.045953 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.046048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.046192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.046280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.046427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.password = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.046505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.046670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.token = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.046748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.046901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.046993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.047139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.047219 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.047390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.047469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.047629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.047717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.047876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.047960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.048116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.048195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.048353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.048438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.048585 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.048668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.048836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.048914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.049129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.049231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.049354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.049439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.049598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.049676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.049847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.049934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.050088 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.050168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.050358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.050436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.050599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.050684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.050845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.050925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.051078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.051157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.051316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.051404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.051564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.051643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.051810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.051890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.052042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.052137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.052289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.052370 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.052536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.052620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.052791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.052879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.053027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.053138 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.053304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.053382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.053540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.053626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.053787 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.053867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.054021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.054106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.054268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.054356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.054509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.054586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.054742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.054820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.054968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.055044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.055213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.055292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.055446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.055525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.055682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.055763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.055931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.056010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.056182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.056260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.056413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.056494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.056643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.056722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.056873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.056956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.057143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.057227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.057383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.057463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.057631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.057710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.057865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.057944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.058102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.058188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.058339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.058418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.058588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.058670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.058822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.058899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.059052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.059130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.059289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.059369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.059519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.059597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.059753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.059831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.059996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.060104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.060234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.060326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.060487 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.060566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.060717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.060799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.060980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.061058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.061245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.061324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.061491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.061572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.061744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.061824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.062001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.062079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.062481 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.062568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.062744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.062830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.062974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.063053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.063198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.063274 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.063437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.063516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.063680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.063758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.063919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.063999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.064157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.064235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.064392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.064469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.064636 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.064717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.064865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.064944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.065120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.065202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.065366 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.065447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.065609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.065687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.065843 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.065920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.066077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.066158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.066312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.066397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.066540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.066620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.066779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.066860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.067032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.067113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.067261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.067346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.067499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.067577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.067741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.067820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.067976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.068062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.068205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.068283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.068437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.068528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.068670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.068758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.068904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.068983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.069162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.069246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.069393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.069489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.069640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.069720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.069873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.069951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.070113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.070201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.070353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.070432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.070594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.070673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.070822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.070908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.071071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.071150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.071307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.071384 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.071544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.071630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.071779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.071858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.072012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.072090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.072252 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.072338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.072494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.072572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.072729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.072808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.072968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.073054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.073222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.073301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.073462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.073558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.073696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.073795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.073936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.074016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.074174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.074253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.074421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.074507 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.074663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.074742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.074896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.074974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.075134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.075229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.075372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.075451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.075614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.075693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.075861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.075946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.076098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.076178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.076332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.076409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.076599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.076690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.076847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.076926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.077109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.077202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.077367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.077480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.077598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.077682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.077830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] role.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.077907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.078077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] role.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.078232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.078309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.078380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.078543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.078623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.078788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.078874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.079033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.079112 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.079273 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] role.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.079351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.079516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.079603 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] role.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.079749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.079828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.079994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.080072 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.080242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.080322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.080479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.080558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.080726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.080811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.080973 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.081054 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.081246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.081316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.081492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.081640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.081746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.081898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.081976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.082224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.082293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.082477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.082560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.082731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.082809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.082980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.083048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.083231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.083383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.083454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.083628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.083887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.083955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.084145 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.084223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.084403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.084483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.084660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.084793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.084901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.085034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.085193 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.085298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.085444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.085691 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.085826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.085931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.086169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.086238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.086404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.086526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.086662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.086773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.086924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.087016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.087177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.087365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.087448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.087607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.087676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.087850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.087918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.088107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.088182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.088346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.088413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.088597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.088838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.088906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.089113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.089188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.089371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.089489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.089609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.089741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.089858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.089991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.090107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.090246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.090361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.090603 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.090682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.090846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.090917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.091085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.091153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.091336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.091413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.091591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.091671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.091822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.092067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.092635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.092890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.093000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.093182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.093348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.093436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.093676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.093745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.093920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.094046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.094169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.094304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.094410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.094652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.094721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.094893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.095062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.095133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.095306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.095375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.095550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.095619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.095793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.095864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.096047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.096163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.096333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.096565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.096725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.096851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.096968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.097076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.097159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.097771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.097929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.098014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.098101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.098172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 18:48:36.098243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.099040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.099288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.100159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.100159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 18:48:36.100159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 18:48:36.129546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.129779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.129929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.130061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.130189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.130373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.138604 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.138833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.166886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.167101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.167262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.167399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.309838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.310097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.362124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.362422 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.362597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.362855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.426382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.426615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.426840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.426966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.427110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.427196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.427273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.427412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.427497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.427635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.427861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.428082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.452757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.452935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.453044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.453200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.453284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.453431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.453515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.453647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.453731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.453864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.453948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.454080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.454160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.454299 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.465068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.465477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.478228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.478452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.478558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.478681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.478765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.478904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.478986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.479121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.479205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.479335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.479417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.479551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.479633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.479765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 18:48:36.496689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.496901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:48:36.497264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.497352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.497469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:48:36.497554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:48:36.497745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.497827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.497995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 18:48:36.498075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.498251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.498332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.498536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.498616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.498795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.499055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.509305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.509583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.509849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.509930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.510127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.510222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.510425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.510673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.518942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.519544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.522258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.522534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.522813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.522893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.523091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.523185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.523372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.523454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.523629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.523711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.523897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.523980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.524168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.524250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.524435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.524515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.524705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.524786 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.524971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.525052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.525268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.525537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.552107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.552652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.553738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.554017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.554304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.554586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.559268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.559659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.559899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.559988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.560260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.560544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.567602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.567884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.568157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.568451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.572398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.572683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.572964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.573046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.573268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.573350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.573550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.573631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.573823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.573904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.574091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.574173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.574355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.574434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.574626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.574898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.594324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.594600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.594854 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.595135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.599204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.599769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.602856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.603256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.603437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.603837 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.608316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.608910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.611953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.612567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.615649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.615949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.616233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.616316 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.616518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.616618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.616798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.616882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.617074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.617179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.617387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.617669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.636180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.636474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.636742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.636888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.637015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.637208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.637480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.637755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.645504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.646209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.647205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.647481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.647915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.648214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.650701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.650983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.651262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.651501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.651582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.651799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.651888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.652102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.652408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.652745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.665680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.665957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.666243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.666523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.666736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.667018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.667307 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.667590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.668056 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.668352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.668647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.669146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.669441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.669735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.687143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.688592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.690709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.691928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.694044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.695262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.695765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.696984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.697421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.697701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.697992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.698670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.698950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.699241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.705441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.706690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.707055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.708318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.710311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.710641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.710979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.711257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.711606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.711899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.712292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.712575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.720876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.721185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.721475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.722328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.722614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.722897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.729350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.729626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.729904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.730184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.730455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.730823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.730912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 18:48:36.731106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.731194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.731377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.731457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.731649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.731729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.731920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.732000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.732189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.732303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.732454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.732594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.732728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.732878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.732999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.733176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.734007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.734285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.734560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.734847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.735110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 18:48:36.771312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fc6d35f9c0 pid: 80157 (default app) Jun 21 18:48:36.772914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fc6d35f9c0 pid: 80158 (default app) Jun 21 18:48:36.774737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd277de-db78-41c1-b8b8-ec6deecb7c70 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:36.775035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd277de-db78-41c1-b8b8-ec6deecb7c70 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:36.775318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd277de-db78-41c1-b8b8-ec6deecb7c70 None None] PATH_INFO: `/v3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:36.779484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1/1] 10.222.0.70 () {50 vars in 791 bytes} [Tue Jun 21 18:48:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:44.351019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f45b-9bb6-458a-aad3-e008767b8413 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:44.351575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f45b-9bb6-458a-aad3-e008767b8413 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:44.351575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f45b-9bb6-458a-aad3-e008767b8413 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:44.355832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1/2] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:48:44.361265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:44.361654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:44.361831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:44.431278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: INFO dbcounter [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] Registered counter for database keystone Jun 21 18:48:44.453436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG oslo_db.sqlalchemy.engines [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80157) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 18:48:44.459610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writer thread running {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 18:48:44.637810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG passlib.handlers.bcrypt [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80157) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 18:48:44.638153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG passlib.handlers.bcrypt [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80157) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 18:48:44.692633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:44.694280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:44.815027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 2/3] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 21 18:48:44.976434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7f45b-9bb6-458a-aad3-e008767b8413 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:45.047158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: INFO dbcounter [None req-96c7f45b-9bb6-458a-aad3-e008767b8413 None None] Registered counter for database keystone Jun 21 18:48:45.066327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG oslo_db.sqlalchemy.engines [None req-96c7f45b-9bb6-458a-aad3-e008767b8413 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80158) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 18:48:45.072535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writer thread running {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 18:48:45.192260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7f45b-9bb6-458a-aad3-e008767b8413 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:45.196587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a13be3-c9e9-4f6b-98d8-bf81c028f597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:45.197276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a13be3-c9e9-4f6b-98d8-bf81c028f597 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:45.197570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a13be3-c9e9-4f6b-98d8-bf81c028f597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:45.197853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a13be3-c9e9-4f6b-98d8-bf81c028f597 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:45.206081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a13be3-c9e9-4f6b-98d8-bf81c028f597 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:45.258714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WARNING keystone.common.rbac_enforcer.enforcer [None req-79a13be3-c9e9-4f6b-98d8-bf81c028f597 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 18:48:45.259005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a13be3-c9e9-4f6b-98d8-bf81c028f597 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:45.269204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 2/4] 10.222.0.70 () {58 vars in 1324 bytes} [Tue Jun 21 18:48:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:48:45.275040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:45.289672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcac3db-b616-49c2-98d8-7f0a702ba2c7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:45.294028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3556b34-4c14-434c-a861-76108d4c83d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:45.295774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3556b34-4c14-434c-a861-76108d4c83d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:45.296060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3556b34-4c14-434c-a861-76108d4c83d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:45.296337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3556b34-4c14-434c-a861-76108d4c83d0 admin admin] PATH_INFO: `/v3/projects/8dc534a786d340398da9cb4c7a7e5355` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:45.300430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3556b34-4c14-434c-a861-76108d4c83d0 admin admin] RBAC: Authorizing `identity:get_project(project_id=8dc534a786d340398da9cb4c7a7e5355)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:45.352410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c3556b34-4c14-434c-a861-76108d4c83d0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 18:48:45.352754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3556b34-4c14-434c-a861-76108d4c83d0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:45.357417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 3/5] 10.222.0.70 () {56 vars in 1207 bytes} [Tue Jun 21 18:48:45 2022] GET /identity/v3/projects/8dc534a786d340398da9cb4c7a7e5355 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:47.322413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9826c8b-d702-477d-81d0-95b4d3dc6e40 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:47.322940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9826c8b-d702-477d-81d0-95b4d3dc6e40 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:47.323482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9826c8b-d702-477d-81d0-95b4d3dc6e40 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:47.327692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 3/6] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:47.333111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f057-8d1d-4082-bb39-8cf2569af4b8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:47.333589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f057-8d1d-4082-bb39-8cf2569af4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:47.333690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f057-8d1d-4082-bb39-8cf2569af4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:47.352018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-07a5f057-8d1d-4082-bb39-8cf2569af4b8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:47.353552 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-07a5f057-8d1d-4082-bb39-8cf2569af4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:47.361344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 4/7] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 21 18:48:47.383802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9826c8b-d702-477d-81d0-95b4d3dc6e40 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:47.396465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9826c8b-d702-477d-81d0-95b4d3dc6e40 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:47.400705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453bc7d-d6ea-4c43-825c-e189eac1b593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:47.401362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453bc7d-d6ea-4c43-825c-e189eac1b593 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:47.401653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453bc7d-d6ea-4c43-825c-e189eac1b593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:47.401937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453bc7d-d6ea-4c43-825c-e189eac1b593 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:47.409614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453bc7d-d6ea-4c43-825c-e189eac1b593 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:47.411391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453bc7d-d6ea-4c43-825c-e189eac1b593 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:47.420132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 4/8] 10.222.0.70 () {58 vars in 1324 bytes} [Tue Jun 21 18:48:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:48:47.425326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f057-8d1d-4082-bb39-8cf2569af4b8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:47.434294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f057-8d1d-4082-bb39-8cf2569af4b8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:47.439463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df7d62-a1c7-4ebb-9e79-2421fb7a5be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:47.440317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7d62-a1c7-4ebb-9e79-2421fb7a5be3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:47.440606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7d62-a1c7-4ebb-9e79-2421fb7a5be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:47.440888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7d62-a1c7-4ebb-9e79-2421fb7a5be3 admin admin] PATH_INFO: `/v3/users/8dcd62677ffc4535814a24f589fce966` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:47.442373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df7d62-a1c7-4ebb-9e79-2421fb7a5be3 admin admin] RBAC: Authorizing `identity:get_user(user_id=8dcd62677ffc4535814a24f589fce966)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:47.444212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df7d62-a1c7-4ebb-9e79-2421fb7a5be3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:47.445832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 5/9] 10.222.0.70 () {56 vars in 1198 bytes} [Tue Jun 21 18:48:47 2022] GET /identity/v3/users/8dcd62677ffc4535814a24f589fce966 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:49.513969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196ac1-21a9-44bc-9a59-4031d827f7b7 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:49.514486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196ac1-21a9-44bc-9a59-4031d827f7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:49.514593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196ac1-21a9-44bc-9a59-4031d827f7b7 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:49.515866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 5/10] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:49.520886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6824b-40ab-48c3-a6e2-ff20335ce211 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:49.521322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6824b-40ab-48c3-a6e2-ff20335ce211 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:49.521632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6824b-40ab-48c3-a6e2-ff20335ce211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:49.538652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-36d6824b-40ab-48c3-a6e2-ff20335ce211 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:49.539843 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-36d6824b-40ab-48c3-a6e2-ff20335ce211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:49.549295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d691d-d755-4d86-b2e4-b6510e2c8c46 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:49.549597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d691d-d755-4d86-b2e4-b6510e2c8c46 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:49.549886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d691d-d755-4d86-b2e4-b6510e2c8c46 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:49.551230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 6/11] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:48:49.556372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b71d2-6891-482c-b71a-51e6949c6b4a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:49.556727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b71d2-6891-482c-b71a-51e6949c6b4a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:49.557109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b71d2-6891-482c-b71a-51e6949c6b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:49.596974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG passlib.handlers.bcrypt [None req-ab9b71d2-6891-482c-b71a-51e6949c6b4a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80158) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 18:48:49.597362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG passlib.handlers.bcrypt [None req-ab9b71d2-6891-482c-b71a-51e6949c6b4a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80158) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 18:48:49.622652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 6/12] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:48:49.642749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ab9b71d2-6891-482c-b71a-51e6949c6b4a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:49.644342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ab9b71d2-6891-482c-b71a-51e6949c6b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:49.645461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6824b-40ab-48c3-a6e2-ff20335ce211 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:49.668970 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:49.743959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 7/13] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:48:49.749335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6824b-40ab-48c3-a6e2-ff20335ce211 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:49.765657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b71d2-6891-482c-b71a-51e6949c6b4a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:49.782731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642eab1-d3ee-4f8e-9c33-d8c97b76324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:49.783526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642eab1-d3ee-4f8e-9c33-d8c97b76324f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:49.783814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642eab1-d3ee-4f8e-9c33-d8c97b76324f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:49.784095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642eab1-d3ee-4f8e-9c33-d8c97b76324f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:49.795956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642eab1-d3ee-4f8e-9c33-d8c97b76324f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5642eab1-d3ee-4f8e-9c33-d8c97b76324f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:48:49.807145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:48:49.808208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:48:49.810064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 7/14] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:48:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:48:49.814390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642eab1-d3ee-4f8e-9c33-d8c97b76324f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:49.871795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b71d2-6891-482c-b71a-51e6949c6b4a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:49.874374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:49.895150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:49.911341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c31c77-755a-41f9-8d07-1d1a9917bb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:49.912243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c31c77-755a-41f9-8d07-1d1a9917bb79 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:49.912537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c31c77-755a-41f9-8d07-1d1a9917bb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:49.912820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c31c77-755a-41f9-8d07-1d1a9917bb79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:49.914412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642eab1-d3ee-4f8e-9c33-d8c97b76324f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:49.949002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc1584-3d19-49bb-8588-8c4e4a507090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:49.949694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc1584-3d19-49bb-8588-8c4e4a507090 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:49.949985 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc1584-3d19-49bb-8588-8c4e4a507090 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:49.950268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc1584-3d19-49bb-8588-8c4e4a507090 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:49.951522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc1584-3d19-49bb-8588-8c4e4a507090 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:48:49.951840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc1584-3d19-49bb-8588-8c4e4a507090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:49.953295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc1584-3d19-49bb-8588-8c4e4a507090 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:49.964870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 8/15] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:48:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:48:49.971089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc1584-3d19-49bb-8588-8c4e4a507090 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:49.980673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c31c77-755a-41f9-8d07-1d1a9917bb79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:49.982264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c31c77-755a-41f9-8d07-1d1a9917bb79 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:50.068926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc1584-3d19-49bb-8588-8c4e4a507090 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:50.078218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 8/16] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:48:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:48:50.083849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c31c77-755a-41f9-8d07-1d1a9917bb79 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:50.086759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:50.103375 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b696d5e-a5dd-41a7-883b-39599b5ed18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:50.104254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b696d5e-a5dd-41a7-883b-39599b5ed18c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:50.104546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b696d5e-a5dd-41a7-883b-39599b5ed18c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:50.104832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b696d5e-a5dd-41a7-883b-39599b5ed18c None admin] PATH_INFO: `/v3/users/8dcd62677ffc4535814a24f589fce966` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:50.106080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b696d5e-a5dd-41a7-883b-39599b5ed18c None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd62677ffc4535814a24f589fce966)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:50.107572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b696d5e-a5dd-41a7-883b-39599b5ed18c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:50.109230 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 9/17] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:48:49 2022] GET /identity/v3/users/8dcd62677ffc4535814a24f589fce966 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:48:50.114682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b696d5e-a5dd-41a7-883b-39599b5ed18c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:50.128561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:50.183772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c31c77-755a-41f9-8d07-1d1a9917bb79 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:50.218765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b696d5e-a5dd-41a7-883b-39599b5ed18c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:50.219038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef22c07-8b64-4433-a38f-64ba800f6ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:50.219776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef22c07-8b64-4433-a38f-64ba800f6ceb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:50.220176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef22c07-8b64-4433-a38f-64ba800f6ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:50.220337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef22c07-8b64-4433-a38f-64ba800f6ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:50.242087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef22c07-8b64-4433-a38f-64ba800f6ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:50.243549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef22c07-8b64-4433-a38f-64ba800f6ceb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:50.252670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf41d7-80ca-441e-8bb7-2e2dc7486577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:50.253641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf41d7-80ca-441e-8bb7-2e2dc7486577 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:50.253924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf41d7-80ca-441e-8bb7-2e2dc7486577 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:50.254201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf41d7-80ca-441e-8bb7-2e2dc7486577 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:50.255406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf41d7-80ca-441e-8bb7-2e2dc7486577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:50.256837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf41d7-80ca-441e-8bb7-2e2dc7486577 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:50.258484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 10/18] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:50.264733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf41d7-80ca-441e-8bb7-2e2dc7486577 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-8ef22c07-8b64-4433-a38f-64ba800f6ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:48:50.265022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:48:50.265826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:48:50.266618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:48:50.266618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:48:50.266618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:48:50.266618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:48:50.266618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 9/19] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:48:50.271826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef22c07-8b64-4433-a38f-64ba800f6ceb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:50.327639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:50.350028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:50.369645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf41d7-80ca-441e-8bb7-2e2dc7486577 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:50.373769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef22c07-8b64-4433-a38f-64ba800f6ceb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:50.404011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7be99-47a7-400a-8ab5-6bed67adf0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:50.404621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7be99-47a7-400a-8ab5-6bed67adf0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:50.404910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7be99-47a7-400a-8ab5-6bed67adf0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:50.405225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7be99-47a7-400a-8ab5-6bed67adf0c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:50.406070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7be99-47a7-400a-8ab5-6bed67adf0c4 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.domain.id=default, user.id=8dcd62677ffc4535814a24f589fce966) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:48:50.406388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7be99-47a7-400a-8ab5-6bed67adf0c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:50.407647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e8841-0a2e-4a7b-8dee-3e1361e5697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:50.407840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7be99-47a7-400a-8ab5-6bed67adf0c4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:50.408309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e8841-0a2e-4a7b-8dee-3e1361e5697e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:50.408598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e8841-0a2e-4a7b-8dee-3e1361e5697e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:50.408878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e8841-0a2e-4a7b-8dee-3e1361e5697e None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:50.409645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e8841-0a2e-4a7b-8dee-3e1361e5697e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:48:50.409962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e8841-0a2e-4a7b-8dee-3e1361e5697e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:50.411381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e8841-0a2e-4a7b-8dee-3e1361e5697e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:50.414690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 11/20] 10.222.0.70 () {56 vars in 1322 bytes} [Tue Jun 21 18:48:50 2022] GET /identity/v3/role_assignments?user.id=8dcd62677ffc4535814a24f589fce966&scope.domain.id=default&role.id=e089a0d6040040c4bcf143a680488e3d => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:48:50.430207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 10/21] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:48:52.680258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad604d-1966-48b0-be86-eaa80a0bc4d2 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:52.681308 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad604d-1966-48b0-be86-eaa80a0bc4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:52.681802 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad604d-1966-48b0-be86-eaa80a0bc4d2 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:52.683672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 12/22] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:52.689517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c227fa4-69ef-4bd6-be47-424ec91334be None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:52.690352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c227fa4-69ef-4bd6-be47-424ec91334be None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:52.690830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c227fa4-69ef-4bd6-be47-424ec91334be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:52.714327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4c227fa4-69ef-4bd6-be47-424ec91334be None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:52.715547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4c227fa4-69ef-4bd6-be47-424ec91334be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:52.791755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 11/23] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:48:52.815958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad604d-1966-48b0-be86-eaa80a0bc4d2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:52.906003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90d6d4-0701-4610-b6e2-b7c8275573fa None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:52.906314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90d6d4-0701-4610-b6e2-b7c8275573fa None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:52.906579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90d6d4-0701-4610-b6e2-b7c8275573fa None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:52.908243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 12/24] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:52.914657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77046d3f-67f6-449d-9202-700bc7258ea1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:52.914949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77046d3f-67f6-449d-9202-700bc7258ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:52.915238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77046d3f-67f6-449d-9202-700bc7258ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:52.942335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-77046d3f-67f6-449d-9202-700bc7258ea1 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:52.943551 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-77046d3f-67f6-449d-9202-700bc7258ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:52.991211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad604d-1966-48b0-be86-eaa80a0bc4d2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:53.025013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:53.058714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd52de2-be40-4439-a869-868ef57519ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:53.059528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd52de2-be40-4439-a869-868ef57519ef None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.059816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd52de2-be40-4439-a869-868ef57519ef None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.060098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd52de2-be40-4439-a869-868ef57519ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.079446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 13/25] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:48:53.082534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd52de2-be40-4439-a869-868ef57519ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:53.084292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd52de2-be40-4439-a869-868ef57519ef None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:53.087430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ffd52de2-be40-4439-a869-868ef57519ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:48:53.105120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:48:53.105992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:48:53.106791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:48:53.106791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:48:53.106791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:48:53.106791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:48:53.109276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 13/26] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:48:53.117944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77046d3f-67f6-449d-9202-700bc7258ea1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:53.122723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd52de2-be40-4439-a869-868ef57519ef None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:53.305396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd52de2-be40-4439-a869-868ef57519ef None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:53.323451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77046d3f-67f6-449d-9202-700bc7258ea1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:53.353560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda70e3a-8b5c-454b-a386-8ec9dd869894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:53.354684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda70e3a-8b5c-454b-a386-8ec9dd869894 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.355174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda70e3a-8b5c-454b-a386-8ec9dd869894 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.355654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda70e3a-8b5c-454b-a386-8ec9dd869894 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.367528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda70e3a-8b5c-454b-a386-8ec9dd869894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-fda70e3a-8b5c-454b-a386-8ec9dd869894 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:48:53.377128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:48:53.377989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:48:53.378760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:48:53.381772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 14/27] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:48:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:48:53.383335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eab906-22e9-4ac5-a773-96c7d485ddc0 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.383847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eab906-22e9-4ac5-a773-96c7d485ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.384328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eab906-22e9-4ac5-a773-96c7d485ddc0 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.386989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 15/28] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:53.389149 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eab906-22e9-4ac5-a773-96c7d485ddc0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:53.415091 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e16c5a-03f4-4c09-8871-ac012b1a10bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:53.415760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e16c5a-03f4-4c09-8871-ac012b1a10bc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.416045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e16c5a-03f4-4c09-8871-ac012b1a10bc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.416319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e16c5a-03f4-4c09-8871-ac012b1a10bc None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.417044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e16c5a-03f4-4c09-8871-ac012b1a10bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:48:53.417378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e16c5a-03f4-4c09-8871-ac012b1a10bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:53.418764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e16c5a-03f4-4c09-8871-ac012b1a10bc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:53.440767 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 14/29] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:53.442128 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72939dc-78d4-48ef-8dce-6a858c3e6613 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.442424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72939dc-78d4-48ef-8dce-6a858c3e6613 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.442708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72939dc-78d4-48ef-8dce-6a858c3e6613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.474109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f72939dc-78d4-48ef-8dce-6a858c3e6613 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:53.475322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f72939dc-78d4-48ef-8dce-6a858c3e6613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:53.500868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:53.515927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:53.560017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eab906-22e9-4ac5-a773-96c7d485ddc0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:53.563392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 15/30] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:48:53.564052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72939dc-78d4-48ef-8dce-6a858c3e6613 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:53.598640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd01987-e09e-4eff-a236-562919ac1635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:53.599608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd01987-e09e-4eff-a236-562919ac1635 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.600082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd01987-e09e-4eff-a236-562919ac1635 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.600518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd01987-e09e-4eff-a236-562919ac1635 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.601463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd01987-e09e-4eff-a236-562919ac1635 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:48:53.601961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd01987-e09e-4eff-a236-562919ac1635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:53.603488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd01987-e09e-4eff-a236-562919ac1635 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:53.614414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 16/31] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:48:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:53.615934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecb71d-c8da-471c-b8f6-71c515d4c363 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.616440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecb71d-c8da-471c-b8f6-71c515d4c363 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.616940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecb71d-c8da-471c-b8f6-71c515d4c363 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.618753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 17/32] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:53.624001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecb71d-c8da-471c-b8f6-71c515d4c363 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:53.671891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72939dc-78d4-48ef-8dce-6a858c3e6613 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:53.708941 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d5ade-be28-4983-bb62-8d4f07906040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:53.709647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d5ade-be28-4983-bb62-8d4f07906040 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.709939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d5ade-be28-4983-bb62-8d4f07906040 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.710223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d5ade-be28-4983-bb62-8d4f07906040 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.711329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d5ade-be28-4983-bb62-8d4f07906040 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:53.712907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d5ade-be28-4983-bb62-8d4f07906040 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:53.738420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecb71d-c8da-471c-b8f6-71c515d4c363 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:53.758578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:53.777255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26070c-1cdc-42f6-a206-21bb4f54ed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:53.778199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26070c-1cdc-42f6-a206-21bb4f54ed8a None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.778636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26070c-1cdc-42f6-a206-21bb4f54ed8a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.779090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26070c-1cdc-42f6-a206-21bb4f54ed8a None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.780081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26070c-1cdc-42f6-a206-21bb4f54ed8a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:53.781670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26070c-1cdc-42f6-a206-21bb4f54ed8a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:53.863458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 16/33] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:48:53 2022] POST /identity/v3/projects => generated 301 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:48:53.864417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d5ade-be28-4983-bb62-8d4f07906040 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:53.886664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:53.916245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 18/34] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jun 21 18:48:53 2022] POST /identity/v3/roles => generated 219 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:48:53.918184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459751a0-51b2-4bcb-85f4-5b21dd4bc69d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:53.918680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459751a0-51b2-4bcb-85f4-5b21dd4bc69d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:53.919237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459751a0-51b2-4bcb-85f4-5b21dd4bc69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:53.940761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-459751a0-51b2-4bcb-85f4-5b21dd4bc69d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:53.942849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-459751a0-51b2-4bcb-85f4-5b21dd4bc69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:53.984686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d5ade-be28-4983-bb62-8d4f07906040 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:54.025218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 19/35] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:48:54.046759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642953a3-8e4c-445e-ab86-3851f6674b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:54.048098 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642953a3-8e4c-445e-ab86-3851f6674b14 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.048592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642953a3-8e4c-445e-ab86-3851f6674b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.049068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642953a3-8e4c-445e-ab86-3851f6674b14 None admin] PATH_INFO: `/v3/users/8dcd62677ffc4535814a24f589fce966` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.051163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642953a3-8e4c-445e-ab86-3851f6674b14 None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd62677ffc4535814a24f589fce966)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:54.053063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642953a3-8e4c-445e-ab86-3851f6674b14 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:54.055620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 17/36] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:48:53 2022] GET /identity/v3/users/8dcd62677ffc4535814a24f589fce966 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:54.062749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459751a0-51b2-4bcb-85f4-5b21dd4bc69d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:54.109554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642953a3-8e4c-445e-ab86-3851f6674b14 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:54.223912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459751a0-51b2-4bcb-85f4-5b21dd4bc69d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:54.236045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:54.265330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:54.275035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0f51e-a96d-4cad-a8d7-91f1f64c732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:54.276108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0f51e-a96d-4cad-a8d7-91f1f64c732e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.276681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0f51e-a96d-4cad-a8d7-91f1f64c732e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.277298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0f51e-a96d-4cad-a8d7-91f1f64c732e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.279302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0f51e-a96d-4cad-a8d7-91f1f64c732e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:54.281293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0f51e-a96d-4cad-a8d7-91f1f64c732e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:54.283400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642953a3-8e4c-445e-ab86-3851f6674b14 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:54.283567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 20/37] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:48:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:54.290444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8bf5f-74d4-4c24-a8ff-1c6cc499eb60 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.290444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8bf5f-74d4-4c24-a8ff-1c6cc499eb60 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.290444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8bf5f-74d4-4c24-a8ff-1c6cc499eb60 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.290444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 21/38] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:54.294113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8bf5f-74d4-4c24-a8ff-1c6cc499eb60 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:54.331001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7f431-ea3b-464c-ac65-494fcac57aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:54.331685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7f431-ea3b-464c-ac65-494fcac57aec None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.331976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7f431-ea3b-464c-ac65-494fcac57aec None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.332262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7f431-ea3b-464c-ac65-494fcac57aec None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.333027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7f431-ea3b-464c-ac65-494fcac57aec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:54.334454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7f431-ea3b-464c-ac65-494fcac57aec None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:54.357642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 18/39] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jun 21 18:48:54 2022] POST /identity/v3/roles => generated 225 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:48:54.357642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc78e0-5c6f-4006-b96d-45414452b29b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.357642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc78e0-5c6f-4006-b96d-45414452b29b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.357642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc78e0-5c6f-4006-b96d-45414452b29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.376785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e4fc78e0-5c6f-4006-b96d-45414452b29b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:54.378778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e4fc78e0-5c6f-4006-b96d-45414452b29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:54.430661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8bf5f-74d4-4c24-a8ff-1c6cc499eb60 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:54.473223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 19/40] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:48:54.474732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cf9d2-32f7-4f48-b04f-9cca84cfbe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:54.475628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cf9d2-32f7-4f48-b04f-9cca84cfbe78 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.475991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cf9d2-32f7-4f48-b04f-9cca84cfbe78 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.476340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cf9d2-32f7-4f48-b04f-9cca84cfbe78 None admin] PATH_INFO: `/v3/domains/default/users/8dcd62677ffc4535814a24f589fce966/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.483281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cf9d2-32f7-4f48-b04f-9cca84cfbe78 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8dcd62677ffc4535814a24f589fce966, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:54.483281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cf9d2-32f7-4f48-b04f-9cca84cfbe78 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:54.496958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc78e0-5c6f-4006-b96d-45414452b29b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:54.499943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 22/41] 10.222.0.70 () {58 vars in 1361 bytes} [Tue Jun 21 18:48:54 2022] PUT /identity/v3/domains/default/users/8dcd62677ffc4535814a24f589fce966/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:48:54.556429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc75e1f-85af-4ace-8323-eb240a5cedc9 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.556736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc75e1f-85af-4ace-8323-eb240a5cedc9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.557019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc75e1f-85af-4ace-8323-eb240a5cedc9 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.558568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 23/42] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:48:54.565776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a94aed-e713-48c3-b6fd-214356f44363 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.566081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a94aed-e713-48c3-b6fd-214356f44363 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.566346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a94aed-e713-48c3-b6fd-214356f44363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.587482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-25a94aed-e713-48c3-b6fd-214356f44363 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:54.588726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-25a94aed-e713-48c3-b6fd-214356f44363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:54.600825 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:54.631944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:54.633274 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc78e0-5c6f-4006-b96d-45414452b29b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:54.675596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73059b9b-8f67-4ef9-b43f-535359449b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:54.676301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73059b9b-8f67-4ef9-b43f-535359449b0e None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.676591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73059b9b-8f67-4ef9-b43f-535359449b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.676872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73059b9b-8f67-4ef9-b43f-535359449b0e None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.677717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73059b9b-8f67-4ef9-b43f-535359449b0e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:54.679169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73059b9b-8f67-4ef9-b43f-535359449b0e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:54.696183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 20/43] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jun 21 18:48:54 2022] POST /identity/v3/roles => generated 223 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:48:54.790907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 24/44] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:48:54.814653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73059b9b-8f67-4ef9-b43f-535359449b0e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:54.923386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73059b9b-8f67-4ef9-b43f-535359449b0e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:54.956793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80c735-9914-432c-b224-bbf27072c98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:54.957615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80c735-9914-432c-b224-bbf27072c98b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:54.957907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80c735-9914-432c-b224-bbf27072c98b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:54.958190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80c735-9914-432c-b224-bbf27072c98b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:54.959359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80c735-9914-432c-b224-bbf27072c98b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:54.960805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80c735-9914-432c-b224-bbf27072c98b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:54.962477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 21/45] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:48:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:54.968540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a94aed-e713-48c3-b6fd-214356f44363 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:55.068478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a94aed-e713-48c3-b6fd-214356f44363 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:55.102992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb326593-ba11-4fb4-acf2-f12d1843022f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:55.103654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326593-ba11-4fb4-acf2-f12d1843022f None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:55.103946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326593-ba11-4fb4-acf2-f12d1843022f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:55.104231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326593-ba11-4fb4-acf2-f12d1843022f None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:55.104948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb326593-ba11-4fb4-acf2-f12d1843022f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:55.106444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb326593-ba11-4fb4-acf2-f12d1843022f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:55.132074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 25/46] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:48:54 2022] POST /identity/v3/projects => generated 312 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:48:56.556726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab637c8f-1cae-49e8-879f-1eecee1da0b0 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:56.557255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab637c8f-1cae-49e8-879f-1eecee1da0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:56.557327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab637c8f-1cae-49e8-879f-1eecee1da0b0 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:56.558821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 22/47] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:56.564398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ebef-4213-4b16-b77f-8d987bacfc02 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:56.564824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ebef-4213-4b16-b77f-8d987bacfc02 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:56.564969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ebef-4213-4b16-b77f-8d987bacfc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:56.582432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8637ebef-4213-4b16-b77f-8d987bacfc02 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:56.583537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8637ebef-4213-4b16-b77f-8d987bacfc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:56.597060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:56.649324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 26/48] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:48:56.672825 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab637c8f-1cae-49e8-879f-1eecee1da0b0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:56.676665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:56.773479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab637c8f-1cae-49e8-879f-1eecee1da0b0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:56.806277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99643bde-0b76-4345-a496-6a698024e472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:56.807067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99643bde-0b76-4345-a496-6a698024e472 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:56.807358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99643bde-0b76-4345-a496-6a698024e472 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:56.807643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99643bde-0b76-4345-a496-6a698024e472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:56.819446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99643bde-0b76-4345-a496-6a698024e472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-99643bde-0b76-4345-a496-6a698024e472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:48:56.828718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:48:56.829578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:48:56.830563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 23/49] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:48:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:48:56.835753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ebef-4213-4b16-b77f-8d987bacfc02 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:56.933563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ebef-4213-4b16-b77f-8d987bacfc02 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:56.968032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80441b01-7cd8-4199-860a-a908a18dbb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:56.968700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80441b01-7cd8-4199-860a-a908a18dbb8e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:56.968988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80441b01-7cd8-4199-860a-a908a18dbb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:56.969297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80441b01-7cd8-4199-860a-a908a18dbb8e None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:56.970051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80441b01-7cd8-4199-860a-a908a18dbb8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:48:56.970364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80441b01-7cd8-4199-860a-a908a18dbb8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:56.971760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80441b01-7cd8-4199-860a-a908a18dbb8e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:56.979796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 27/50] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:48:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:56.985283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99643bde-0b76-4345-a496-6a698024e472 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:57.046648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:57.085894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99643bde-0b76-4345-a496-6a698024e472 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:57.118392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c93721-dcda-4023-a23e-7d574f11e2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:57.119261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c93721-dcda-4023-a23e-7d574f11e2ba None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:57.119551 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c93721-dcda-4023-a23e-7d574f11e2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:57.119835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c93721-dcda-4023-a23e-7d574f11e2ba None admin] PATH_INFO: `/v3/users/8dcd62677ffc4535814a24f589fce966` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:57.121045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c93721-dcda-4023-a23e-7d574f11e2ba None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd62677ffc4535814a24f589fce966)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:57.122533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c93721-dcda-4023-a23e-7d574f11e2ba None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:57.124706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 24/51] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:48:56 2022] GET /identity/v3/users/8dcd62677ffc4535814a24f589fce966 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:57.130104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80441b01-7cd8-4199-860a-a908a18dbb8e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:57.143366 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:57.161064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed5df2-7ae5-4126-93f0-2c79803e7257 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:57.161391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed5df2-7ae5-4126-93f0-2c79803e7257 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:57.161681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed5df2-7ae5-4126-93f0-2c79803e7257 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:57.163045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 25/52] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:57.167804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3555c0-3283-4b18-b5eb-e34e8bfb6800 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:57.168096 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3555c0-3283-4b18-b5eb-e34e8bfb6800 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:57.168379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3555c0-3283-4b18-b5eb-e34e8bfb6800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:57.184565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-eb3555c0-3283-4b18-b5eb-e34e8bfb6800 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:57.186294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-eb3555c0-3283-4b18-b5eb-e34e8bfb6800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:57.231254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80441b01-7cd8-4199-860a-a908a18dbb8e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:57.253098 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 26/53] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:48:57.265489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b7c6-82ea-4ef9-be26-8c9da0c9eb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:57.266279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b7c6-82ea-4ef9-be26-8c9da0c9eb20 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:57.266572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b7c6-82ea-4ef9-be26-8c9da0c9eb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:57.266856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b7c6-82ea-4ef9-be26-8c9da0c9eb20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:57.268037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558b7c6-82ea-4ef9-be26-8c9da0c9eb20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:57.269531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558b7c6-82ea-4ef9-be26-8c9da0c9eb20 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:57.271171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 28/54] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:48:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:48:57.276705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3555c0-3283-4b18-b5eb-e34e8bfb6800 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:57.276882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b7c6-82ea-4ef9-be26-8c9da0c9eb20 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:57.360664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:57.384872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b7c6-82ea-4ef9-be26-8c9da0c9eb20 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:57.420361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d08f4-82a5-473e-b733-ab3729acba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:57.421023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d08f4-82a5-473e-b733-ab3729acba5b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:57.421343 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d08f4-82a5-473e-b733-ab3729acba5b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:57.421633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d08f4-82a5-473e-b733-ab3729acba5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:57.422474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d08f4-82a5-473e-b733-ab3729acba5b None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.domain.id=default, user.id=8dcd62677ffc4535814a24f589fce966) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:48:57.422798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d08f4-82a5-473e-b733-ab3729acba5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:57.424287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d08f4-82a5-473e-b733-ab3729acba5b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:57.430195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 29/55] 10.222.0.70 () {56 vars in 1322 bytes} [Tue Jun 21 18:48:57 2022] GET /identity/v3/role_assignments?user.id=8dcd62677ffc4535814a24f589fce966&scope.domain.id=default&role.id=e089a0d6040040c4bcf143a680488e3d => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:48:57.443561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:57.488258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3555c0-3283-4b18-b5eb-e34e8bfb6800 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:57.525657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eaaf8-a5b8-4336-84a2-f26335b208e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:57.526470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eaaf8-a5b8-4336-84a2-f26335b208e6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:57.526762 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eaaf8-a5b8-4336-84a2-f26335b208e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:57.527049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eaaf8-a5b8-4336-84a2-f26335b208e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:57.528270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eaaf8-a5b8-4336-84a2-f26335b208e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:57.529829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eaaf8-a5b8-4336-84a2-f26335b208e6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:57.531511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 27/56] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:48:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:57.537282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d08f4-82a5-473e-b733-ab3729acba5b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:57.640879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d08f4-82a5-473e-b733-ab3729acba5b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:57.673973 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7c43e-e61d-467b-ba91-e66b924e1f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:57.674626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7c43e-e61d-467b-ba91-e66b924e1f07 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:57.674912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7c43e-e61d-467b-ba91-e66b924e1f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:57.675188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7c43e-e61d-467b-ba91-e66b924e1f07 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:57.675902 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7c43e-e61d-467b-ba91-e66b924e1f07 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:57.677358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7c43e-e61d-467b-ba91-e66b924e1f07 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:57.690444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:57.701443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 30/57] 10.222.0.70 () {60 vars in 1140 bytes} [Tue Jun 21 18:48:57 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:48:59.729242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd10db-2277-45ba-a8c9-95855cfc67ca None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:59.729745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd10db-2277-45ba-a8c9-95855cfc67ca None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:59.729827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd10db-2277-45ba-a8c9-95855cfc67ca None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:59.731191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 28/58] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:48:59.736835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98004f-1281-42d8-9639-efd2ad767d32 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:59.737269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98004f-1281-42d8-9639-efd2ad767d32 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:59.737458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98004f-1281-42d8-9639-efd2ad767d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:59.754451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7c98004f-1281-42d8-9639-efd2ad767d32 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:48:59.755660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7c98004f-1281-42d8-9639-efd2ad767d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:48:59.820732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 31/59] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:48:59.844920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd10db-2277-45ba-a8c9-95855cfc67ca None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:48:59.947283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd10db-2277-45ba-a8c9-95855cfc67ca None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:48:59.977328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:48:59.983695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0b4b8-1ccf-45e0-9b61-7b81e75f0c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:48:59.984447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0b4b8-1ccf-45e0-9b61-7b81e75f0c4e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:48:59.985055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0b4b8-1ccf-45e0-9b61-7b81e75f0c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:48:59.985560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0b4b8-1ccf-45e0-9b61-7b81e75f0c4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:48:59.986874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0b4b8-1ccf-45e0-9b61-7b81e75f0c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:48:59.988359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0b4b8-1ccf-45e0-9b61-7b81e75f0c4e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:48:59.990066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 29/60] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:48:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:48:59.995601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98004f-1281-42d8-9639-efd2ad767d32 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:00.095647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98004f-1281-42d8-9639-efd2ad767d32 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:00.118975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:00.130272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee54d7-3322-4b87-91ae-7f20e108c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:00.130947 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee54d7-3322-4b87-91ae-7f20e108c76f None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:00.131237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee54d7-3322-4b87-91ae-7f20e108c76f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:00.131518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee54d7-3322-4b87-91ae-7f20e108c76f None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:00.132248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee54d7-3322-4b87-91ae-7f20e108c76f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:00.133744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee54d7-3322-4b87-91ae-7f20e108c76f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:00.161480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 32/61] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:48:59 2022] POST /identity/v3/users => generated 275 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:03.240136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f117f54-2a19-40d9-a98c-356a74f75d6f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.240675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f117f54-2a19-40d9-a98c-356a74f75d6f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.240751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f117f54-2a19-40d9-a98c-356a74f75d6f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.242383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 30/62] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:03.248329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c556c62-974f-4482-9d86-442b3d8feeaf None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.248733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c556c62-974f-4482-9d86-442b3d8feeaf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.248915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c556c62-974f-4482-9d86-442b3d8feeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.273044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8c556c62-974f-4482-9d86-442b3d8feeaf None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:03.274284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8c556c62-974f-4482-9d86-442b3d8feeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:03.352944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196d683-76a1-4c0a-9d49-a921e785242d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.353295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196d683-76a1-4c0a-9d49-a921e785242d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.353568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196d683-76a1-4c0a-9d49-a921e785242d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.354974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 31/63] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:03.367193 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56d5ac-3d4b-4915-92ae-73d3b9d87bd2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.367678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56d5ac-3d4b-4915-92ae-73d3b9d87bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.368104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56d5ac-3d4b-4915-92ae-73d3b9d87bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.369722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 33/64] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:03.393599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c556c62-974f-4482-9d86-442b3d8feeaf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:03.399925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0f56d5ac-3d4b-4915-92ae-73d3b9d87bd2 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:03.405939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0f56d5ac-3d4b-4915-92ae-73d3b9d87bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:03.543400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 32/65] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:03.557414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c556c62-974f-4482-9d86-442b3d8feeaf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:03.577215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:03.581598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56d5ac-3d4b-4915-92ae-73d3b9d87bd2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:03.613187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266418b4-143b-464e-a26d-855c7eb7304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:03.614226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266418b4-143b-464e-a26d-855c7eb7304f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.614619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266418b4-143b-464e-a26d-855c7eb7304f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.615028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266418b4-143b-464e-a26d-855c7eb7304f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.641653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266418b4-143b-464e-a26d-855c7eb7304f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-266418b4-143b-464e-a26d-855c7eb7304f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:03.658827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:03.659695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:03.660523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:03.663288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 34/66] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:03.668672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266418b4-143b-464e-a26d-855c7eb7304f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:03.690836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56d5ac-3d4b-4915-92ae-73d3b9d87bd2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:03.720626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:03.730963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678e8ab-a7ed-4368-b2d4-814c404493e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:03.732557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678e8ab-a7ed-4368-b2d4-814c404493e2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.733019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678e8ab-a7ed-4368-b2d4-814c404493e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.733521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678e8ab-a7ed-4368-b2d4-814c404493e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.744420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678e8ab-a7ed-4368-b2d4-814c404493e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-4678e8ab-a7ed-4368-b2d4-814c404493e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:03.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:03.753291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:03.754070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:03.756432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 33/67] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:49:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:03.757873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e020b-c411-4e90-9b44-273f16f09e0a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.758321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e020b-c411-4e90-9b44-273f16f09e0a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.758759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e020b-c411-4e90-9b44-273f16f09e0a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.760183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 34/68] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:03.761972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e020b-c411-4e90-9b44-273f16f09e0a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:03.809134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266418b4-143b-464e-a26d-855c7eb7304f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:03.847558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d5eda-3634-4ce9-b8ab-2a26b34bb18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:03.848509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d5eda-3634-4ce9-b8ab-2a26b34bb18b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.848948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d5eda-3634-4ce9-b8ab-2a26b34bb18b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.849952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d5eda-3634-4ce9-b8ab-2a26b34bb18b None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.850091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d5eda-3634-4ce9-b8ab-2a26b34bb18b None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:03.850322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d5eda-3634-4ce9-b8ab-2a26b34bb18b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:03.851720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d5eda-3634-4ce9-b8ab-2a26b34bb18b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:03.861258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 35/69] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:03.862036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89ef89-bb97-4898-8b22-0cbc8b1873c2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.862402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89ef89-bb97-4898-8b22-0cbc8b1873c2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.862657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89ef89-bb97-4898-8b22-0cbc8b1873c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.880402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6d89ef89-bb97-4898-8b22-0cbc8b1873c2 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:03.881633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6d89ef89-bb97-4898-8b22-0cbc8b1873c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:03.882451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e020b-c411-4e90-9b44-273f16f09e0a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:03.910017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:03.920000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2a3ba-fcd0-4937-a42b-c33fc56b45a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:03.920677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2a3ba-fcd0-4937-a42b-c33fc56b45a0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.921074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2a3ba-fcd0-4937-a42b-c33fc56b45a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.921389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2a3ba-fcd0-4937-a42b-c33fc56b45a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.922146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2a3ba-fcd0-4937-a42b-c33fc56b45a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:03.922460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2a3ba-fcd0-4937-a42b-c33fc56b45a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:03.923850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2a3ba-fcd0-4937-a42b-c33fc56b45a0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:03.936941 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 35/70] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:03.941812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19573972-77e3-467a-9e77-5908a9daf48e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:03.941812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19573972-77e3-467a-9e77-5908a9daf48e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:03.941812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19573972-77e3-467a-9e77-5908a9daf48e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:03.941812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 36/71] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:03.941812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19573972-77e3-467a-9e77-5908a9daf48e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:03.970971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 36/72] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:03.970971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89ef89-bb97-4898-8b22-0cbc8b1873c2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.000805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:04.052400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19573972-77e3-467a-9e77-5908a9daf48e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:04.069776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89ef89-bb97-4898-8b22-0cbc8b1873c2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:04.087106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2e147-6d6f-4f1d-bf52-e86811ee467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:04.087972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2e147-6d6f-4f1d-bf52-e86811ee467f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.088264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2e147-6d6f-4f1d-bf52-e86811ee467f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.088546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2e147-6d6f-4f1d-bf52-e86811ee467f None admin] PATH_INFO: `/v3/users/d809509539f742ddb1d244a1c6f9e2af` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.103747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf3a49-b216-4b8c-96f2-b57d86348856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:04.103882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2e147-6d6f-4f1d-bf52-e86811ee467f None admin] RBAC: Authorizing `identity:get_user(user_id=d809509539f742ddb1d244a1c6f9e2af)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:04.104611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf3a49-b216-4b8c-96f2-b57d86348856 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.104896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf3a49-b216-4b8c-96f2-b57d86348856 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.105203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf3a49-b216-4b8c-96f2-b57d86348856 None admin] PATH_INFO: `/v3/users/8dcd62677ffc4535814a24f589fce966` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.105391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2e147-6d6f-4f1d-bf52-e86811ee467f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:04.106409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf3a49-b216-4b8c-96f2-b57d86348856 None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd62677ffc4535814a24f589fce966)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:04.107097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 37/73] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:03 2022] GET /identity/v3/users/d809509539f742ddb1d244a1c6f9e2af => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:04.107877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf3a49-b216-4b8c-96f2-b57d86348856 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:04.108262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da213971-7a87-4f0d-b153-51df248a1427 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.108556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da213971-7a87-4f0d-b153-51df248a1427 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.108842 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da213971-7a87-4f0d-b153-51df248a1427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.109761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 37/74] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:03 2022] GET /identity/v3/users/8dcd62677ffc4535814a24f589fce966 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:04.111023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666800c-11ad-474f-be0f-8b3f56b0d30a None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.111621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666800c-11ad-474f-be0f-8b3f56b0d30a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.111735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666800c-11ad-474f-be0f-8b3f56b0d30a None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.112935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 38/75] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:04.116726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666800c-11ad-474f-be0f-8b3f56b0d30a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.131588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-da213971-7a87-4f0d-b153-51df248a1427 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:04.132777 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-da213971-7a87-4f0d-b153-51df248a1427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:04.143666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:04.198626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 38/76] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:04.199809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da213971-7a87-4f0d-b153-51df248a1427 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.218808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666800c-11ad-474f-be0f-8b3f56b0d30a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:04.241058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:04.253141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d260fa1-ecd8-42e0-a1ea-81eaf1147330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:04.253954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d260fa1-ecd8-42e0-a1ea-81eaf1147330 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.254245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d260fa1-ecd8-42e0-a1ea-81eaf1147330 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.254526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d260fa1-ecd8-42e0-a1ea-81eaf1147330 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.265290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d260fa1-ecd8-42e0-a1ea-81eaf1147330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-0d260fa1-ecd8-42e0-a1ea-81eaf1147330 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:04.273447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:04.274449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:04.275414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 39/77] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:04.276157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d260fa1-ecd8-42e0-a1ea-81eaf1147330 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.301839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da213971-7a87-4f0d-b153-51df248a1427 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:04.340792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28a8ab-ca5b-439c-9420-fd8325a64c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:04.342380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28a8ab-ca5b-439c-9420-fd8325a64c58 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.342711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28a8ab-ca5b-439c-9420-fd8325a64c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.342997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28a8ab-ca5b-439c-9420-fd8325a64c58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.373272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:04.382813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d260fa1-ecd8-42e0-a1ea-81eaf1147330 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:04.411736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28a8ab-ca5b-439c-9420-fd8325a64c58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:04.413225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28a8ab-ca5b-439c-9420-fd8325a64c58 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:04.416623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092802b-6ab3-4df2-bbb7-b7f7201f015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:04.417296 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092802b-6ab3-4df2-bbb7-b7f7201f015a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.417587 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092802b-6ab3-4df2-bbb7-b7f7201f015a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.417868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092802b-6ab3-4df2-bbb7-b7f7201f015a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.450550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:04.484742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092802b-6ab3-4df2-bbb7-b7f7201f015a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:04.486267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092802b-6ab3-4df2-bbb7-b7f7201f015a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:04.511074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 39/78] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:04.512335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dea224-88f1-46b3-8b84-be7c2e3c6755 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.512634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dea224-88f1-46b3-8b84-be7c2e3c6755 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.512914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dea224-88f1-46b3-8b84-be7c2e3c6755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.528974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-45dea224-88f1-46b3-8b84-be7c2e3c6755 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:04.530199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-45dea224-88f1-46b3-8b84-be7c2e3c6755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:04.577795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:04.588541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 40/79] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:04.589974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092802b-6ab3-4df2-bbb7-b7f7201f015a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.598380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 40/80] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:04.599211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dea224-88f1-46b3-8b84-be7c2e3c6755 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.671600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:04.705845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092802b-6ab3-4df2-bbb7-b7f7201f015a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:04.706460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dea224-88f1-46b3-8b84-be7c2e3c6755 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:04.741590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd961c-0ac1-4cc2-8b56-f276aceb0695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:04.741813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9890a9-3215-455d-88c2-b44b2a4f732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:04.742324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9890a9-3215-455d-88c2-b44b2a4f732d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.742416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd961c-0ac1-4cc2-8b56-f276aceb0695 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.742611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9890a9-3215-455d-88c2-b44b2a4f732d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.742698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd961c-0ac1-4cc2-8b56-f276aceb0695 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.742892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9890a9-3215-455d-88c2-b44b2a4f732d None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.742975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd961c-0ac1-4cc2-8b56-f276aceb0695 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.743653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9890a9-3215-455d-88c2-b44b2a4f732d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:04.743964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9890a9-3215-455d-88c2-b44b2a4f732d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:04.744139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd961c-0ac1-4cc2-8b56-f276aceb0695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:04.745401 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9890a9-3215-455d-88c2-b44b2a4f732d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:04.745657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd961c-0ac1-4cc2-8b56-f276aceb0695 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:04.747346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 41/81] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:04.748683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd961c-0ac1-4cc2-8b56-f276aceb0695 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.759198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 41/82] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:04.760051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9890a9-3215-455d-88c2-b44b2a4f732d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.810707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:04.851815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd961c-0ac1-4cc2-8b56-f276aceb0695 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:04.861636 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9890a9-3215-455d-88c2-b44b2a4f732d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:04.885796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782ba86-3709-494e-9210-68db29cee0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:04.886658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782ba86-3709-494e-9210-68db29cee0ab None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.886954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782ba86-3709-494e-9210-68db29cee0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.887229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782ba86-3709-494e-9210-68db29cee0ab None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.888449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782ba86-3709-494e-9210-68db29cee0ab None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:04.889528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:04.890178 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782ba86-3709-494e-9210-68db29cee0ab None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:04.896538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 42/83] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:04 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:04.896538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782ba86-3709-494e-9210-68db29cee0ab None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.896746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacd365-3949-434d-9d20-672a80fdd273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:04.897152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacd365-3949-434d-9d20-672a80fdd273 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:04.897690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacd365-3949-434d-9d20-672a80fdd273 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:04.898197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacd365-3949-434d-9d20-672a80fdd273 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:04.899642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacd365-3949-434d-9d20-672a80fdd273 None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:04.901530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacd365-3949-434d-9d20-672a80fdd273 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:04.903748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 42/84] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:04 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:04.904641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacd365-3949-434d-9d20-672a80fdd273 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:04.992068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782ba86-3709-494e-9210-68db29cee0ab None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.004709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacd365-3949-434d-9d20-672a80fdd273 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.021227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:05.030153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5cca2-3865-46b9-860a-9f5cc2b3a24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:05.030947 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5cca2-3865-46b9-860a-9f5cc2b3a24e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:05.031245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5cca2-3865-46b9-860a-9f5cc2b3a24e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:05.031526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5cca2-3865-46b9-860a-9f5cc2b3a24e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:05.038666 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d643e6-c036-45c3-a60e-ea017ac6ae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:05.039333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d643e6-c036-45c3-a60e-ea017ac6ae1b None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:05.039624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d643e6-c036-45c3-a60e-ea017ac6ae1b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:05.039909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d643e6-c036-45c3-a60e-ea017ac6ae1b None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:05.040664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d643e6-c036-45c3-a60e-ea017ac6ae1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:05.041568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5cca2-3865-46b9-860a-9f5cc2b3a24e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:05.042150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d643e6-c036-45c3-a60e-ea017ac6ae1b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-dbd5cca2-3865-46b9-860a-9f5cc2b3a24e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:05.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:05.050713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:05.051456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 43/85] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:05.052083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5cca2-3865-46b9-860a-9f5cc2b3a24e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:05.074434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 43/86] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:04 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:05.075266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d643e6-c036-45c3-a60e-ea017ac6ae1b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:05.141350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:05.169809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5cca2-3865-46b9-860a-9f5cc2b3a24e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.193925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d643e6-c036-45c3-a60e-ea017ac6ae1b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.209346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c3d92-f388-4544-9fa8-57a6fdd66d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:05.210307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c3d92-f388-4544-9fa8-57a6fdd66d66 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:05.210666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c3d92-f388-4544-9fa8-57a6fdd66d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:05.211013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c3d92-f388-4544-9fa8-57a6fdd66d66 None admin] PATH_INFO: `/v3/users/d809509539f742ddb1d244a1c6f9e2af` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:05.212310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c3d92-f388-4544-9fa8-57a6fdd66d66 None admin] RBAC: Authorizing `identity:get_user(user_id=d809509539f742ddb1d244a1c6f9e2af)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:05.213879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c3d92-f388-4544-9fa8-57a6fdd66d66 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:05.219992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 44/87] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:05 2022] GET /identity/v3/users/d809509539f742ddb1d244a1c6f9e2af => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:05.219992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c3d92-f388-4544-9fa8-57a6fdd66d66 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:05.238375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230f221-6c08-4335-8c01-cd8c341ddcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:05.239233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230f221-6c08-4335-8c01-cd8c341ddcfd None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:05.239855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230f221-6c08-4335-8c01-cd8c341ddcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:05.240148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230f221-6c08-4335-8c01-cd8c341ddcfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:05.241778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230f221-6c08-4335-8c01-cd8c341ddcfd None admin] RBAC: Adding query filter params (role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=26049e1a44a94f1ab013063cd9767d44, user.id=d809509539f742ddb1d244a1c6f9e2af) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:05.241778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230f221-6c08-4335-8c01-cd8c341ddcfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:05.242759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230f221-6c08-4335-8c01-cd8c341ddcfd None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:05.253156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 44/88] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:05 2022] GET /identity/v3/role_assignments?user.id=d809509539f742ddb1d244a1c6f9e2af&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=301682fc5c92440f874912eabfed6f70 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:05.253966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230f221-6c08-4335-8c01-cd8c341ddcfd None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:05.293211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:05.375335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c3d92-f388-4544-9fa8-57a6fdd66d66 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.396763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230f221-6c08-4335-8c01-cd8c341ddcfd None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.425483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:05.435724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131b807-c250-4261-97eb-8234015c329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:05.436390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131b807-c250-4261-97eb-8234015c329e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:05.436685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131b807-c250-4261-97eb-8234015c329e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:05.436966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131b807-c250-4261-97eb-8234015c329e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:05.438090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131b807-c250-4261-97eb-8234015c329e None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=26049e1a44a94f1ab013063cd9767d44, user.id=8dcd62677ffc4535814a24f589fce966) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:05.438408 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131b807-c250-4261-97eb-8234015c329e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:05.439885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131b807-c250-4261-97eb-8234015c329e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:05.446671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 45/89] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:05 2022] GET /identity/v3/role_assignments?user.id=8dcd62677ffc4535814a24f589fce966&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=e089a0d6040040c4bcf143a680488e3d => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:05.447291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131b807-c250-4261-97eb-8234015c329e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:05.453021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bc393-8a10-4e17-948e-5c82fe0041f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:05.453801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bc393-8a10-4e17-948e-5c82fe0041f7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:05.454092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bc393-8a10-4e17-948e-5c82fe0041f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:05.454377 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bc393-8a10-4e17-948e-5c82fe0041f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:05.455154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bc393-8a10-4e17-948e-5c82fe0041f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:05.455473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bc393-8a10-4e17-948e-5c82fe0041f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:05.456915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bc393-8a10-4e17-948e-5c82fe0041f7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:05.469409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 45/90] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:05.475177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bc393-8a10-4e17-948e-5c82fe0041f7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:05.573855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131b807-c250-4261-97eb-8234015c329e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.588889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bc393-8a10-4e17-948e-5c82fe0041f7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.592404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:05.619541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c1ba4-02fe-45f9-a313-f6696d506cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:05.620219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c1ba4-02fe-45f9-a313-f6696d506cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:05.620510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c1ba4-02fe-45f9-a313-f6696d506cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:05.620887 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c1ba4-02fe-45f9-a313-f6696d506cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:05.668255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c991287-d842-41fb-be04-ad5a938ebbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:05.669145 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c991287-d842-41fb-be04-ad5a938ebbcc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:05.669995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c991287-d842-41fb-be04-ad5a938ebbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:05.670330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c991287-d842-41fb-be04-ad5a938ebbcc None admin] PATH_INFO: `/v3/users/d809509539f742ddb1d244a1c6f9e2af` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:05.673527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c991287-d842-41fb-be04-ad5a938ebbcc None admin] RBAC: Authorizing `identity:get_user(user_id=d809509539f742ddb1d244a1c6f9e2af)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:05.677195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c991287-d842-41fb-be04-ad5a938ebbcc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:05.679339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 46/91] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:05 2022] GET /identity/v3/users/d809509539f742ddb1d244a1c6f9e2af => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:05.702981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c991287-d842-41fb-be04-ad5a938ebbcc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:05.732485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c1ba4-02fe-45f9-a313-f6696d506cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:05.734016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c1ba4-02fe-45f9-a313-f6696d506cb7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:05.774609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:05.839838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 46/92] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:05.845448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c1ba4-02fe-45f9-a313-f6696d506cb7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:05.878785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:05.920728 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c991287-d842-41fb-be04-ad5a938ebbcc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.953433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c1ba4-02fe-45f9-a313-f6696d506cb7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:05.990546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e573bb3-0a9e-491a-af00-d1b112da7d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:05.991425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e573bb3-0a9e-491a-af00-d1b112da7d0d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:05.991717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e573bb3-0a9e-491a-af00-d1b112da7d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:05.992005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e573bb3-0a9e-491a-af00-d1b112da7d0d None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:05.993344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e573bb3-0a9e-491a-af00-d1b112da7d0d None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:05.994887 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e573bb3-0a9e-491a-af00-d1b112da7d0d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:05.997184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 47/93] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:05 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:06.002967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e573bb3-0a9e-491a-af00-d1b112da7d0d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:06.017896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb4136-98a5-4006-a406-2082e7dae279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:06.018582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb4136-98a5-4006-a406-2082e7dae279 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:06.018875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb4136-98a5-4006-a406-2082e7dae279 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:06.019162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb4136-98a5-4006-a406-2082e7dae279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:06.096172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:06.112276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e573bb3-0a9e-491a-af00-d1b112da7d0d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:06.149493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a7c35-3596-46e1-827b-38153f264ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:06.150163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a7c35-3596-46e1-827b-38153f264ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:06.150455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a7c35-3596-46e1-827b-38153f264ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:06.150748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a7c35-3596-46e1-827b-38153f264ee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:06.151600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a7c35-3596-46e1-827b-38153f264ee7 None admin] RBAC: Adding query filter params (role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=26049e1a44a94f1ab013063cd9767d44, user.id=d809509539f742ddb1d244a1c6f9e2af) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:06.151986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a7c35-3596-46e1-827b-38153f264ee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:06.154638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a7c35-3596-46e1-827b-38153f264ee7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:06.160888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 48/94] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:06 2022] GET /identity/v3/role_assignments?user.id=d809509539f742ddb1d244a1c6f9e2af&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=8184510545604582b267e4d1871ae9bf => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:06.207492 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb4136-98a5-4006-a406-2082e7dae279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:06.208993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb4136-98a5-4006-a406-2082e7dae279 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:06.292510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:06.364644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 47/95] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:49:06.371400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a7c35-3596-46e1-827b-38153f264ee7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:06.482252 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a7c35-3596-46e1-827b-38153f264ee7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:06.518238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744da3c-44f9-44b4-b775-491adb87d355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:06.519440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744da3c-44f9-44b4-b775-491adb87d355 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:06.519949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744da3c-44f9-44b4-b775-491adb87d355 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:06.520430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744da3c-44f9-44b4-b775-491adb87d355 None admin] PATH_INFO: `/v3/projects/5de4e47053624c3ebc24ad72778c18ac` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:06.521966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744da3c-44f9-44b4-b775-491adb87d355 None admin] RBAC: Authorizing `identity:get_project(project_id=5de4e47053624c3ebc24ad72778c18ac)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:06.523682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744da3c-44f9-44b4-b775-491adb87d355 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:06.525987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 49/96] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:06 2022] GET /identity/v3/projects/5de4e47053624c3ebc24ad72778c18ac => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:06.534195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb4136-98a5-4006-a406-2082e7dae279 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:06.639961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb4136-98a5-4006-a406-2082e7dae279 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:06.684074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340a52b-ef3b-4892-88f7-85b3b4fd945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:06.685168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340a52b-ef3b-4892-88f7-85b3b4fd945e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:06.685683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340a52b-ef3b-4892-88f7-85b3b4fd945e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:06.686166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340a52b-ef3b-4892-88f7-85b3b4fd945e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:06.687205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340a52b-ef3b-4892-88f7-85b3b4fd945e None admin] RBAC: Adding query filter params (role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=5de4e47053624c3ebc24ad72778c18ac, user.id=d809509539f742ddb1d244a1c6f9e2af) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:06.688003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340a52b-ef3b-4892-88f7-85b3b4fd945e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:06.689701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340a52b-ef3b-4892-88f7-85b3b4fd945e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:06.695873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 48/97] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:06 2022] GET /identity/v3/role_assignments?user.id=d809509539f742ddb1d244a1c6f9e2af&scope.project.id=5de4e47053624c3ebc24ad72778c18ac&role.id=301682fc5c92440f874912eabfed6f70 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:07.943070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f5b8d-b799-4999-aac7-f8a6c8ee5a72 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:07.944123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f5b8d-b799-4999-aac7-f8a6c8ee5a72 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:07.944623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f5b8d-b799-4999-aac7-f8a6c8ee5a72 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:07.946353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 50/98] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:07.951984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7310299-7884-4375-965e-498808892dc7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:07.952404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7310299-7884-4375-965e-498808892dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:07.952543 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7310299-7884-4375-965e-498808892dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:07.978123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f7310299-7884-4375-965e-498808892dc7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:07.979565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f7310299-7884-4375-965e-498808892dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:07.988533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:08.074566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 49/99] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:08.110627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f5b8d-b799-4999-aac7-f8a6c8ee5a72 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:08.137821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:08.238795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f5b8d-b799-4999-aac7-f8a6c8ee5a72 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:08.286340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db924af-2aba-4162-9fec-18dabb6eb8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:08.287146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db924af-2aba-4162-9fec-18dabb6eb8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:08.287439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db924af-2aba-4162-9fec-18dabb6eb8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:08.287725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db924af-2aba-4162-9fec-18dabb6eb8d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:08.303060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db924af-2aba-4162-9fec-18dabb6eb8d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-8db924af-2aba-4162-9fec-18dabb6eb8d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:08.310799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:08.311637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:08.312641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 51/100] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:08.322229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7310299-7884-4375-965e-498808892dc7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:08.492214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7310299-7884-4375-965e-498808892dc7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:08.495872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bcde1-f38d-428c-a790-aed980ced788 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:08.496444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bcde1-f38d-428c-a790-aed980ced788 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:08.496978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bcde1-f38d-428c-a790-aed980ced788 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:08.498642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 52/101] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:08.505135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fb980-04cb-4c3c-b417-454bfd5dc7b6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:08.506223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fb980-04cb-4c3c-b417-454bfd5dc7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:08.506726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fb980-04cb-4c3c-b417-454bfd5dc7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:08.543594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-159fb980-04cb-4c3c-b417-454bfd5dc7b6 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:08.545454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-159fb980-04cb-4c3c-b417-454bfd5dc7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:08.564217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c47e8f-de95-4a14-9510-ea3c70e8d440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:08.564914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c47e8f-de95-4a14-9510-ea3c70e8d440 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:08.565237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c47e8f-de95-4a14-9510-ea3c70e8d440 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:08.565524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c47e8f-de95-4a14-9510-ea3c70e8d440 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:08.566369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c47e8f-de95-4a14-9510-ea3c70e8d440 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:08.566741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c47e8f-de95-4a14-9510-ea3c70e8d440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:08.568208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c47e8f-de95-4a14-9510-ea3c70e8d440 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:08.585563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 50/102] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:08.589631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20651b87-7c43-4615-94a3-a283ed0dd867 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:08.590371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20651b87-7c43-4615-94a3-a283ed0dd867 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:08.590863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20651b87-7c43-4615-94a3-a283ed0dd867 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:08.593042 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 51/103] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:08.594071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20651b87-7c43-4615-94a3-a283ed0dd867 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:08.606356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:08.718484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20651b87-7c43-4615-94a3-a283ed0dd867 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:08.726669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 53/104] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:08.727807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71190b37-5189-44d4-8c7b-3f39ecac50bc None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:08.728103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71190b37-5189-44d4-8c7b-3f39ecac50bc None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:08.728387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71190b37-5189-44d4-8c7b-3f39ecac50bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:08.758197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302e9fc-b5aa-48ab-9add-4efd90402ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:08.759755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302e9fc-b5aa-48ab-9add-4efd90402ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:08.760257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302e9fc-b5aa-48ab-9add-4efd90402ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:08.760736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302e9fc-b5aa-48ab-9add-4efd90402ba0 None admin] PATH_INFO: `/v3/users/d809509539f742ddb1d244a1c6f9e2af` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:08.762278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302e9fc-b5aa-48ab-9add-4efd90402ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=d809509539f742ddb1d244a1c6f9e2af)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:08.764054 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302e9fc-b5aa-48ab-9add-4efd90402ba0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:08.766020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 52/105] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:08 2022] GET /identity/v3/users/d809509539f742ddb1d244a1c6f9e2af => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:08.774139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-71190b37-5189-44d4-8c7b-3f39ecac50bc None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:08.774516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302e9fc-b5aa-48ab-9add-4efd90402ba0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:08.775362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-71190b37-5189-44d4-8c7b-3f39ecac50bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:08.777916 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:08.864933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:08.875557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 54/106] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:08.876900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71190b37-5189-44d4-8c7b-3f39ecac50bc None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:08.905295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302e9fc-b5aa-48ab-9add-4efd90402ba0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:08.944665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1a9ec-53a2-4d5b-a520-12c6ed5ced5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:08.946090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a9ec-53a2-4d5b-a520-12c6ed5ced5c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:08.946630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a9ec-53a2-4d5b-a520-12c6ed5ced5c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:08.947126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a9ec-53a2-4d5b-a520-12c6ed5ced5c None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:08.948760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1a9ec-53a2-4d5b-a520-12c6ed5ced5c None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:08.950647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1a9ec-53a2-4d5b-a520-12c6ed5ced5c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:08.953446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 53/107] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:08 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:08.955398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70733bf-4201-4862-8660-371d8f76acf8 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:08.955942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70733bf-4201-4862-8660-371d8f76acf8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:08.956421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70733bf-4201-4862-8660-371d8f76acf8 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:08.958091 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 54/108] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:08.959100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70733bf-4201-4862-8660-371d8f76acf8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:09.002027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71190b37-5189-44d4-8c7b-3f39ecac50bc None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:09.040642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41759bef-b555-49c7-82aa-8f08962e3848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:09.041862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759bef-b555-49c7-82aa-8f08962e3848 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.042356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759bef-b555-49c7-82aa-8f08962e3848 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.042890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759bef-b555-49c7-82aa-8f08962e3848 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.051330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:09.057736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41759bef-b555-49c7-82aa-8f08962e3848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-41759bef-b555-49c7-82aa-8f08962e3848 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:09.066403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:09.067305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:09.068083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:09.070504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 55/109] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:49:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:09.071565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41759bef-b555-49c7-82aa-8f08962e3848 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:09.082003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70733bf-4201-4862-8660-371d8f76acf8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:09.126237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f625930-e4bd-4a9b-9e7c-c6929f765f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:09.127047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f625930-e4bd-4a9b-9e7c-c6929f765f71 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.127338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f625930-e4bd-4a9b-9e7c-c6929f765f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.127629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f625930-e4bd-4a9b-9e7c-c6929f765f71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.128853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f625930-e4bd-4a9b-9e7c-c6929f765f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:09.130404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f625930-e4bd-4a9b-9e7c-c6929f765f71 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:09.132092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 55/110] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:09.133474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5e5b8-dc7b-4b35-8266-e02416a5a0f7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.133772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5e5b8-dc7b-4b35-8266-e02416a5a0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.134051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5e5b8-dc7b-4b35-8266-e02416a5a0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.152249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-70e5e5b8-dc7b-4b35-8266-e02416a5a0f7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:09.153521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-70e5e5b8-dc7b-4b35-8266-e02416a5a0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:09.155663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:09.192352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41759bef-b555-49c7-82aa-8f08962e3848 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:09.229302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 56/111] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:09.230165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5e5b8-dc7b-4b35-8266-e02416a5a0f7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:09.231943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087ebc3-28b9-4cfd-90e5-f04edd175bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:09.233071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087ebc3-28b9-4cfd-90e5-f04edd175bc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.234295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087ebc3-28b9-4cfd-90e5-f04edd175bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.234595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087ebc3-28b9-4cfd-90e5-f04edd175bc4 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44/users/d809509539f742ddb1d244a1c6f9e2af/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.236723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087ebc3-28b9-4cfd-90e5-f04edd175bc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=26049e1a44a94f1ab013063cd9767d44, user_id=d809509539f742ddb1d244a1c6f9e2af, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:09.245286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087ebc3-28b9-4cfd-90e5-f04edd175bc4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:09.265971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 56/112] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:49:09 2022] PUT /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44/users/d809509539f742ddb1d244a1c6f9e2af/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:09.265971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004f356-6bd7-4b0e-b335-ca23e2c1630e None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.265971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004f356-6bd7-4b0e-b335-ca23e2c1630e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.265971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004f356-6bd7-4b0e-b335-ca23e2c1630e None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.265971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 57/113] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:09.266823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004f356-6bd7-4b0e-b335-ca23e2c1630e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:09.367848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:09.371170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5e5b8-dc7b-4b35-8266-e02416a5a0f7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:09.413143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24751f-3786-4fd4-805a-e6679ab2aac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:09.414141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24751f-3786-4fd4-805a-e6679ab2aac2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.414323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24751f-3786-4fd4-805a-e6679ab2aac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.414586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24751f-3786-4fd4-805a-e6679ab2aac2 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.415427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24751f-3786-4fd4-805a-e6679ab2aac2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:09.415747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24751f-3786-4fd4-805a-e6679ab2aac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:09.417281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24751f-3786-4fd4-805a-e6679ab2aac2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:09.424962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:09.434527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 57/114] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:09.434773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24751f-3786-4fd4-805a-e6679ab2aac2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:09.452403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004f356-6bd7-4b0e-b335-ca23e2c1630e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:09.491567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5ee01-5b0d-4988-a98c-c8a86f9326ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:09.492358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5ee01-5b0d-4988-a98c-c8a86f9326ae None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.492657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5ee01-5b0d-4988-a98c-c8a86f9326ae None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.492949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5ee01-5b0d-4988-a98c-c8a86f9326ae None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.493718 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5ee01-5b0d-4988-a98c-c8a86f9326ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:09.495334 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5ee01-5b0d-4988-a98c-c8a86f9326ae None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:09.524047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 58/115] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:49:09 2022] POST /identity/v3/projects => generated 302 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:09.525859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d0ee-7871-4d5b-a3c9-5c806d2752d4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.526690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d0ee-7871-4d5b-a3c9-5c806d2752d4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.527185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d0ee-7871-4d5b-a3c9-5c806d2752d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.548443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9e76d0ee-7871-4d5b-a3c9-5c806d2752d4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:09.550139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9e76d0ee-7871-4d5b-a3c9-5c806d2752d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:09.571336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24751f-3786-4fd4-805a-e6679ab2aac2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:09.615878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92130aa-ba74-4241-adbb-2bfc452a0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:09.616626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92130aa-ba74-4241-adbb-2bfc452a0f8a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.616914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92130aa-ba74-4241-adbb-2bfc452a0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.617318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92130aa-ba74-4241-adbb-2bfc452a0f8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.637179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92130aa-ba74-4241-adbb-2bfc452a0f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b92130aa-ba74-4241-adbb-2bfc452a0f8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:09.641135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:09.642047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:09.642834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 58/116] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:09.643563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92130aa-ba74-4241-adbb-2bfc452a0f8a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:09.704093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 59/117] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:09.705384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d0ee-7871-4d5b-a3c9-5c806d2752d4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:09.735190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:09.787698 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92130aa-ba74-4241-adbb-2bfc452a0f8a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:09.789453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:09.833265 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ecf1-febb-47c7-9dbf-a87078560f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:09.834155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ecf1-febb-47c7-9dbf-a87078560f8c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.834444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ecf1-febb-47c7-9dbf-a87078560f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.834715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ecf1-febb-47c7-9dbf-a87078560f8c None admin] PATH_INFO: `/v3/users/8dcd62677ffc4535814a24f589fce966` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.837437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ecf1-febb-47c7-9dbf-a87078560f8c None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd62677ffc4535814a24f589fce966)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:09.838914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ecf1-febb-47c7-9dbf-a87078560f8c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:09.841466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 59/118] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:09 2022] GET /identity/v3/users/8dcd62677ffc4535814a24f589fce966 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:09.842286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ecf1-febb-47c7-9dbf-a87078560f8c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:09.851886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d0ee-7871-4d5b-a3c9-5c806d2752d4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:09.903210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29295a-56cd-4cad-9e8f-858da44cffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:09.904448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29295a-56cd-4cad-9e8f-858da44cffbe None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.904733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29295a-56cd-4cad-9e8f-858da44cffbe None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.905010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29295a-56cd-4cad-9e8f-858da44cffbe None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.905794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29295a-56cd-4cad-9e8f-858da44cffbe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:09.906106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29295a-56cd-4cad-9e8f-858da44cffbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:09.907496 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29295a-56cd-4cad-9e8f-858da44cffbe None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:09.919901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 60/119] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:09.922030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0cbbc-f1a8-454c-b281-17c0885452f9 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:09.922956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0cbbc-f1a8-454c-b281-17c0885452f9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:09.923708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0cbbc-f1a8-454c-b281-17c0885452f9 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:09.925293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 61/120] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:09.930031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0cbbc-f1a8-454c-b281-17c0885452f9 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:09.981328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ecf1-febb-47c7-9dbf-a87078560f8c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:10.014499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:10.025434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9dfbd-6569-4f2b-8ace-4ebcf0da1c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:10.027279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9dfbd-6569-4f2b-8ace-4ebcf0da1c8f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:10.027773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9dfbd-6569-4f2b-8ace-4ebcf0da1c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:10.029250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9dfbd-6569-4f2b-8ace-4ebcf0da1c8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:10.045275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9dfbd-6569-4f2b-8ace-4ebcf0da1c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:10.054586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0cbbc-f1a8-454c-b281-17c0885452f9 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:10.056378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-87f9dfbd-6569-4f2b-8ace-4ebcf0da1c8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:10.059002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:10.059869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:10.060715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:10.063765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 60/121] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:10.064825 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9dfbd-6569-4f2b-8ace-4ebcf0da1c8f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:10.105537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7361ff-6a81-4f21-b93b-0b99adfb0084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:10.106406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7361ff-6a81-4f21-b93b-0b99adfb0084 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:10.106723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7361ff-6a81-4f21-b93b-0b99adfb0084 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:10.107267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7361ff-6a81-4f21-b93b-0b99adfb0084 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:10.108349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7361ff-6a81-4f21-b93b-0b99adfb0084 None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:10.109995 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7361ff-6a81-4f21-b93b-0b99adfb0084 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:10.112023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 62/122] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:09 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:10.113350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cef7cc-410e-4f81-b26b-b2cae1cb0520 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:10.113688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cef7cc-410e-4f81-b26b-b2cae1cb0520 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:10.114033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cef7cc-410e-4f81-b26b-b2cae1cb0520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:10.132933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-06cef7cc-410e-4f81-b26b-b2cae1cb0520 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:10.134338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-06cef7cc-410e-4f81-b26b-b2cae1cb0520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:10.203055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9dfbd-6569-4f2b-8ace-4ebcf0da1c8f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:10.234480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 63/123] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:10.239615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cef7cc-410e-4f81-b26b-b2cae1cb0520 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:10.250371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f266c45-3e07-468e-b596-3b290b35f1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:10.251302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f266c45-3e07-468e-b596-3b290b35f1cf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:10.251631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f266c45-3e07-468e-b596-3b290b35f1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:10.251910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f266c45-3e07-468e-b596-3b290b35f1cf None admin] PATH_INFO: `/v3/users/a18753029f6c4adaa1a4d5f579d67d55` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:10.274439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f266c45-3e07-468e-b596-3b290b35f1cf None admin] RBAC: Authorizing `identity:get_user(user_id=a18753029f6c4adaa1a4d5f579d67d55)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:10.276204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f266c45-3e07-468e-b596-3b290b35f1cf None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:10.278374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 61/124] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:10 2022] GET /identity/v3/users/a18753029f6c4adaa1a4d5f579d67d55 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:10.279805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f266c45-3e07-468e-b596-3b290b35f1cf None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:10.343282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:10.381608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:10.384578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cef7cc-410e-4f81-b26b-b2cae1cb0520 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:10.414076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f266c45-3e07-468e-b596-3b290b35f1cf None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:10.423027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b489da-b409-48d6-9abc-821d5e1e7064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:10.423912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b489da-b409-48d6-9abc-821d5e1e7064 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:10.424231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b489da-b409-48d6-9abc-821d5e1e7064 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:10.424610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b489da-b409-48d6-9abc-821d5e1e7064 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:10.425518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b489da-b409-48d6-9abc-821d5e1e7064 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:10.426191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b489da-b409-48d6-9abc-821d5e1e7064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:10.427761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b489da-b409-48d6-9abc-821d5e1e7064 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:10.443411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 64/125] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:10.444610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b489da-b409-48d6-9abc-821d5e1e7064 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:10.458449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e710e4-fcab-4931-a433-1c6cf9f30a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:10.460093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e710e4-fcab-4931-a433-1c6cf9f30a17 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:10.460522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e710e4-fcab-4931-a433-1c6cf9f30a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:10.460941 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e710e4-fcab-4931-a433-1c6cf9f30a17 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44/users/8dcd62677ffc4535814a24f589fce966/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:10.463346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e710e4-fcab-4931-a433-1c6cf9f30a17 None admin] RBAC: Authorizing `identity:create_grant(project_id=26049e1a44a94f1ab013063cd9767d44, user_id=8dcd62677ffc4535814a24f589fce966, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:10.464963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e710e4-fcab-4931-a433-1c6cf9f30a17 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:10.488621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 62/126] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:49:10 2022] PUT /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44/users/8dcd62677ffc4535814a24f589fce966/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:10.497847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e710e4-fcab-4931-a433-1c6cf9f30a17 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:10.646237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b489da-b409-48d6-9abc-821d5e1e7064 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:10.668280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e710e4-fcab-4931-a433-1c6cf9f30a17 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:10.677780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:10.710165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be616d52-bb63-4191-8bb6-291f0506992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:10.710337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae88283-6bc1-4786-a767-20d888da7a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:10.711109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae88283-6bc1-4786-a767-20d888da7a41 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:10.711400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae88283-6bc1-4786-a767-20d888da7a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:10.711693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae88283-6bc1-4786-a767-20d888da7a41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:10.711872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be616d52-bb63-4191-8bb6-291f0506992a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:10.712523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be616d52-bb63-4191-8bb6-291f0506992a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:10.712992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be616d52-bb63-4191-8bb6-291f0506992a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:10.729671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae88283-6bc1-4786-a767-20d888da7a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-4ae88283-6bc1-4786-a767-20d888da7a41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:10.742068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:10.742964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:10.743743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:10.750479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 65/127] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:10.756735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae88283-6bc1-4786-a767-20d888da7a41 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:10.806718 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:10.816092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be616d52-bb63-4191-8bb6-291f0506992a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:10.818412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be616d52-bb63-4191-8bb6-291f0506992a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:10.964644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae88283-6bc1-4786-a767-20d888da7a41 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:11.004407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 63/128] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:11.006066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be616d52-bb63-4191-8bb6-291f0506992a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:11.014932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a8039-4394-4081-8dec-13cd054f9841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:11.015828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a8039-4394-4081-8dec-13cd054f9841 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.016106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a8039-4394-4081-8dec-13cd054f9841 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.016323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:11.016421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a8039-4394-4081-8dec-13cd054f9841 None admin] PATH_INFO: `/v3/users/d809509539f742ddb1d244a1c6f9e2af` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.020609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a8039-4394-4081-8dec-13cd054f9841 None admin] RBAC: Authorizing `identity:get_user(user_id=d809509539f742ddb1d244a1c6f9e2af)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:11.022126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a8039-4394-4081-8dec-13cd054f9841 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:11.023808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 66/129] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:10 2022] GET /identity/v3/users/d809509539f742ddb1d244a1c6f9e2af => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:11.024584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a8039-4394-4081-8dec-13cd054f9841 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:11.133172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be616d52-bb63-4191-8bb6-291f0506992a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:11.174489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb9393-e34e-4d62-8cd8-18903292478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:11.175502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb9393-e34e-4d62-8cd8-18903292478b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.176014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb9393-e34e-4d62-8cd8-18903292478b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.176489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb9393-e34e-4d62-8cd8-18903292478b None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.177607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb9393-e34e-4d62-8cd8-18903292478b None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:11.178122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb9393-e34e-4d62-8cd8-18903292478b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:11.179803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb9393-e34e-4d62-8cd8-18903292478b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:11.191099 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 64/130] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:11.192203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb9393-e34e-4d62-8cd8-18903292478b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:11.212570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:11.222977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a8039-4394-4081-8dec-13cd054f9841 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:11.272143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:11.279639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764295-b295-4fbf-898a-0b7875786111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:11.280193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764295-b295-4fbf-898a-0b7875786111 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.280485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764295-b295-4fbf-898a-0b7875786111 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.280770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764295-b295-4fbf-898a-0b7875786111 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.282033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b764295-b295-4fbf-898a-0b7875786111 None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:11.283534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b764295-b295-4fbf-898a-0b7875786111 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:11.285484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 67/131] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:11 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:11.286179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764295-b295-4fbf-898a-0b7875786111 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:11.317139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb9393-e34e-4d62-8cd8-18903292478b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:11.355449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f0e04-a210-4bb1-8f92-ca248b7f3bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:11.357301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f0e04-a210-4bb1-8f92-ca248b7f3bcb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.357826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f0e04-a210-4bb1-8f92-ca248b7f3bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.358304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f0e04-a210-4bb1-8f92-ca248b7f3bcb None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.359726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f0e04-a210-4bb1-8f92-ca248b7f3bcb None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:11.361431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f0e04-a210-4bb1-8f92-ca248b7f3bcb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:11.363355 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 65/132] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:11 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:11.364307 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f0e04-a210-4bb1-8f92-ca248b7f3bcb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:11.476599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f0e04-a210-4bb1-8f92-ca248b7f3bcb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:11.495328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764295-b295-4fbf-898a-0b7875786111 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:11.515673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b7d83-6c22-478a-ac70-62720522e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:11.516716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b7d83-6c22-478a-ac70-62720522e98c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.517235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b7d83-6c22-478a-ac70-62720522e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.517715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b7d83-6c22-478a-ac70-62720522e98c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.518755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b7d83-6c22-478a-ac70-62720522e98c None admin] RBAC: Adding query filter params (role.id=f216586d507240c9b31a20c99b9bd21a, scope.project.id=26049e1a44a94f1ab013063cd9767d44, user.id=a18753029f6c4adaa1a4d5f579d67d55) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:11.519317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b7d83-6c22-478a-ac70-62720522e98c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:11.521053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b7d83-6c22-478a-ac70-62720522e98c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:11.524489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:11.530560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 66/133] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:11 2022] GET /identity/v3/role_assignments?user.id=a18753029f6c4adaa1a4d5f579d67d55&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=f216586d507240c9b31a20c99b9bd21a => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:11.531665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b7d83-6c22-478a-ac70-62720522e98c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:11.601885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4e5e5-6d00-482b-af56-a6dd79216f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:11.602788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4e5e5-6d00-482b-af56-a6dd79216f2a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.603089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4e5e5-6d00-482b-af56-a6dd79216f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.603370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4e5e5-6d00-482b-af56-a6dd79216f2a None admin] PATH_INFO: `/v3/users/d809509539f742ddb1d244a1c6f9e2af` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.604675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4e5e5-6d00-482b-af56-a6dd79216f2a None admin] RBAC: Authorizing `identity:get_user(user_id=d809509539f742ddb1d244a1c6f9e2af)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:11.606300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4e5e5-6d00-482b-af56-a6dd79216f2a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:11.607866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 68/134] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:11 2022] GET /identity/v3/users/d809509539f742ddb1d244a1c6f9e2af => generated 275 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:11.613774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4e5e5-6d00-482b-af56-a6dd79216f2a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:11.655661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b7d83-6c22-478a-ac70-62720522e98c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:11.684393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:11.698251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d20f0-93bf-48ad-a96c-7e35b49da5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:11.699543 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d20f0-93bf-48ad-a96c-7e35b49da5db None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.700033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d20f0-93bf-48ad-a96c-7e35b49da5db None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.701041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d20f0-93bf-48ad-a96c-7e35b49da5db None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44/users/d809509539f742ddb1d244a1c6f9e2af/roles/8184510545604582b267e4d1871ae9bf` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.703639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d20f0-93bf-48ad-a96c-7e35b49da5db None admin] RBAC: Authorizing `identity:create_grant(project_id=26049e1a44a94f1ab013063cd9767d44, user_id=d809509539f742ddb1d244a1c6f9e2af, role_id=8184510545604582b267e4d1871ae9bf)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:11.705544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d20f0-93bf-48ad-a96c-7e35b49da5db None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:11.726776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 67/135] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:49:11 2022] PUT /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44/users/d809509539f742ddb1d244a1c6f9e2af/roles/8184510545604582b267e4d1871ae9bf => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:11.730472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e334f-64e4-4cf6-b7c9-54f0f418cdba None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.731784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e334f-64e4-4cf6-b7c9-54f0f418cdba None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.732291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e334f-64e4-4cf6-b7c9-54f0f418cdba None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.734966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 68/136] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:11.741068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350660b-6067-4b55-b7e7-805197eafab7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.741744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350660b-6067-4b55-b7e7-805197eafab7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.742225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350660b-6067-4b55-b7e7-805197eafab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.783468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4350660b-6067-4b55-b7e7-805197eafab7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:11.785313 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4350660b-6067-4b55-b7e7-805197eafab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:11.860854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4e5e5-6d00-482b-af56-a6dd79216f2a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:11.906479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:11.936049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 69/137] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:11.943358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f692b-1904-4080-a87d-28cbcef88f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:11.944261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f692b-1904-4080-a87d-28cbcef88f4e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:11.944554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f692b-1904-4080-a87d-28cbcef88f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:11.944844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f692b-1904-4080-a87d-28cbcef88f4e None admin] PATH_INFO: `/v3/projects/5de4e47053624c3ebc24ad72778c18ac` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:11.946224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f692b-1904-4080-a87d-28cbcef88f4e None admin] RBAC: Authorizing `identity:get_project(project_id=5de4e47053624c3ebc24ad72778c18ac)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:11.947749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f692b-1904-4080-a87d-28cbcef88f4e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:11.949630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 69/138] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:11 2022] GET /identity/v3/projects/5de4e47053624c3ebc24ad72778c18ac => generated 312 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:11.956248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350660b-6067-4b55-b7e7-805197eafab7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:11.986041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f692b-1904-4080-a87d-28cbcef88f4e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:12.070996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350660b-6067-4b55-b7e7-805197eafab7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:12.109071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b43588-fce5-4505-b8f4-c330c48bae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:12.110503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b43588-fce5-4505-b8f4-c330c48bae5b None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:12.111065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b43588-fce5-4505-b8f4-c330c48bae5b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:12.111578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b43588-fce5-4505-b8f4-c330c48bae5b None admin] PATH_INFO: `/v3/projects/5de4e47053624c3ebc24ad72778c18ac/users/d809509539f742ddb1d244a1c6f9e2af/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:12.113864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b43588-fce5-4505-b8f4-c330c48bae5b None admin] RBAC: Authorizing `identity:create_grant(project_id=5de4e47053624c3ebc24ad72778c18ac, user_id=d809509539f742ddb1d244a1c6f9e2af, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:12.115539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b43588-fce5-4505-b8f4-c330c48bae5b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:12.139647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 70/139] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:49:11 2022] PUT /identity/v3/projects/5de4e47053624c3ebc24ad72778c18ac/users/d809509539f742ddb1d244a1c6f9e2af/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:12.163056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76685d09-4a10-4d9b-b078-7c1c558c7c8a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:12.163685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76685d09-4a10-4d9b-b078-7c1c558c7c8a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:12.164173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76685d09-4a10-4d9b-b078-7c1c558c7c8a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:12.166126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 71/140] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:12.181557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994044ae-5898-493c-a8f8-b56c2027902f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:12.182025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994044ae-5898-493c-a8f8-b56c2027902f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:12.182436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:12.183247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994044ae-5898-493c-a8f8-b56c2027902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:12.193348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f692b-1904-4080-a87d-28cbcef88f4e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:12.211999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-994044ae-5898-493c-a8f8-b56c2027902f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:12.213389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-994044ae-5898-493c-a8f8-b56c2027902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:12.282844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dae53-8761-4c46-973f-4db51d8d7887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:12.283590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dae53-8761-4c46-973f-4db51d8d7887 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:12.283888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dae53-8761-4c46-973f-4db51d8d7887 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:12.284177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dae53-8761-4c46-973f-4db51d8d7887 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:12.308381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258dae53-8761-4c46-973f-4db51d8d7887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-258dae53-8761-4c46-973f-4db51d8d7887 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:12.316764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:12.317917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:12.318893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:12.327965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 70/141] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:12.337150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dae53-8761-4c46-973f-4db51d8d7887 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:12.341478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:12.434980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 72/142] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 21 18:49:12.470479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994044ae-5898-493c-a8f8-b56c2027902f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:12.523620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dae53-8761-4c46-973f-4db51d8d7887 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:12.562851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3087c-d1a0-489f-a756-9ca3ed11c6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:12.563448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3087c-d1a0-489f-a756-9ca3ed11c6bd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:12.563975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3087c-d1a0-489f-a756-9ca3ed11c6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:12.564096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3087c-d1a0-489f-a756-9ca3ed11c6bd None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:12.564852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3087c-d1a0-489f-a756-9ca3ed11c6bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:12.565261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3087c-d1a0-489f-a756-9ca3ed11c6bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:12.566759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3087c-d1a0-489f-a756-9ca3ed11c6bd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:12.576642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 71/143] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:12.582181 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3087c-d1a0-489f-a756-9ca3ed11c6bd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:12.590202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994044ae-5898-493c-a8f8-b56c2027902f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:12.607695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:12.629813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2b8c3-17e0-4140-9bbb-77327b9e346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:12.630733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2b8c3-17e0-4140-9bbb-77327b9e346f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:12.631016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2b8c3-17e0-4140-9bbb-77327b9e346f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:12.631296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2b8c3-17e0-4140-9bbb-77327b9e346f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:12.632626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2b8c3-17e0-4140-9bbb-77327b9e346f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:12.634127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2b8c3-17e0-4140-9bbb-77327b9e346f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:12.636092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 73/144] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:12.643192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2b8c3-17e0-4140-9bbb-77327b9e346f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:12.697319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:12.753729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2b8c3-17e0-4140-9bbb-77327b9e346f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:12.790565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d63245-f5ec-490c-a14d-3b207a161343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:12.791250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d63245-f5ec-490c-a14d-3b207a161343 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:12.791548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d63245-f5ec-490c-a14d-3b207a161343 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:12.791827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d63245-f5ec-490c-a14d-3b207a161343 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:12.792550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d63245-f5ec-490c-a14d-3b207a161343 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:12.794138 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d63245-f5ec-490c-a14d-3b207a161343 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:12.805409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3087c-d1a0-489f-a756-9ca3ed11c6bd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:12.823413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 74/145] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:12 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:49:12.857955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd76386-e77f-404d-9f4b-38c5fce3ed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:12.858906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd76386-e77f-404d-9f4b-38c5fce3ed0b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:12.859191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd76386-e77f-404d-9f4b-38c5fce3ed0b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:12.859991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd76386-e77f-404d-9f4b-38c5fce3ed0b None admin] PATH_INFO: `/v3/users/d809509539f742ddb1d244a1c6f9e2af` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:12.861294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd76386-e77f-404d-9f4b-38c5fce3ed0b None admin] RBAC: Authorizing `identity:get_user(user_id=d809509539f742ddb1d244a1c6f9e2af)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:12.862830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd76386-e77f-404d-9f4b-38c5fce3ed0b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:12.864561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 72/146] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:12 2022] GET /identity/v3/users/d809509539f742ddb1d244a1c6f9e2af => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:12.870968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d63245-f5ec-490c-a14d-3b207a161343 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:13.019770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d63245-f5ec-490c-a14d-3b207a161343 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:13.022969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:13.063189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddd0eb-2634-4db8-81ea-3ed82a3713ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:13.063855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddd0eb-2634-4db8-81ea-3ed82a3713ed None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:13.064210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddd0eb-2634-4db8-81ea-3ed82a3713ed None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:13.064494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddd0eb-2634-4db8-81ea-3ed82a3713ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:13.145277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddd0eb-2634-4db8-81ea-3ed82a3713ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:13.147531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddd0eb-2634-4db8-81ea-3ed82a3713ed None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:13.265023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 75/147] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:13.270661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd76386-e77f-404d-9f4b-38c5fce3ed0b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:13.389454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:13.397764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa46d2d-b454-403f-8c26-c787436d39d2 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:13.398258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa46d2d-b454-403f-8c26-c787436d39d2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:13.398697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa46d2d-b454-403f-8c26-c787436d39d2 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:13.400242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 76/148] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:13.409033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0b02-f79f-451d-89cc-801470198a52 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:13.409946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0b02-f79f-451d-89cc-801470198a52 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:13.410442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0b02-f79f-451d-89cc-801470198a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:13.417351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd76386-e77f-404d-9f4b-38c5fce3ed0b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:13.418161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:13.436456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-affc0b02-f79f-451d-89cc-801470198a52 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:13.438748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-affc0b02-f79f-451d-89cc-801470198a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:13.472835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd0d58-8be8-451f-89f3-2eae2e00149e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:13.473951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd0d58-8be8-451f-89f3-2eae2e00149e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:13.474153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd0d58-8be8-451f-89f3-2eae2e00149e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:13.474399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd0d58-8be8-451f-89f3-2eae2e00149e None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:13.475678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd0d58-8be8-451f-89f3-2eae2e00149e None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:13.477295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd0d58-8be8-451f-89f3-2eae2e00149e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:13.479296 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 73/149] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:13 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:13.485588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd0d58-8be8-451f-89f3-2eae2e00149e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:13.558462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 77/150] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:13.598190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc0b02-f79f-451d-89cc-801470198a52 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:13.599625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd0d58-8be8-451f-89f3-2eae2e00149e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:13.641404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab4220-9fe7-470d-a394-f11f6c7ea985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:13.642928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab4220-9fe7-470d-a394-f11f6c7ea985 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:13.643422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab4220-9fe7-470d-a394-f11f6c7ea985 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:13.643943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab4220-9fe7-470d-a394-f11f6c7ea985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:13.645202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab4220-9fe7-470d-a394-f11f6c7ea985 None admin] RBAC: Adding query filter params (role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=26049e1a44a94f1ab013063cd9767d44, user.id=d809509539f742ddb1d244a1c6f9e2af) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:13.645728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab4220-9fe7-470d-a394-f11f6c7ea985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:13.647556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab4220-9fe7-470d-a394-f11f6c7ea985 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:13.655141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 74/151] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:13 2022] GET /identity/v3/role_assignments?user.id=d809509539f742ddb1d244a1c6f9e2af&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=301682fc5c92440f874912eabfed6f70 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:13.777475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc0b02-f79f-451d-89cc-801470198a52 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:13.823189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51695876-3c71-4c54-93bd-91002159194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:13.824000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51695876-3c71-4c54-93bd-91002159194c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:13.824282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51695876-3c71-4c54-93bd-91002159194c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:13.824560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51695876-3c71-4c54-93bd-91002159194c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:13.830930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:13.843796 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51695876-3c71-4c54-93bd-91002159194c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-51695876-3c71-4c54-93bd-91002159194c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:13.851643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:13.852595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:13.853402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 78/152] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:49:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:13.859938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab4220-9fe7-470d-a394-f11f6c7ea985 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:13.909574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:13.998318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab4220-9fe7-470d-a394-f11f6c7ea985 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:14.041728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63526b6-57eb-4dff-8fb3-c90fd2552f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:14.042836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63526b6-57eb-4dff-8fb3-c90fd2552f52 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:14.043323 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63526b6-57eb-4dff-8fb3-c90fd2552f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:14.043813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63526b6-57eb-4dff-8fb3-c90fd2552f52 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:14.044822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63526b6-57eb-4dff-8fb3-c90fd2552f52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:14.045371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63526b6-57eb-4dff-8fb3-c90fd2552f52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:14.047255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63526b6-57eb-4dff-8fb3-c90fd2552f52 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:14.056698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 75/153] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:14.062669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51695876-3c71-4c54-93bd-91002159194c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:14.192494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51695876-3c71-4c54-93bd-91002159194c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:14.231288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79561924-b831-4274-b162-c466e7b75481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:14.232272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79561924-b831-4274-b162-c466e7b75481 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:14.232562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79561924-b831-4274-b162-c466e7b75481 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:14.232840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79561924-b831-4274-b162-c466e7b75481 None admin] PATH_INFO: `/v3/users/8dcd62677ffc4535814a24f589fce966` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:14.234123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79561924-b831-4274-b162-c466e7b75481 None admin] RBAC: Authorizing `identity:get_user(user_id=8dcd62677ffc4535814a24f589fce966)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:14.235576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79561924-b831-4274-b162-c466e7b75481 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:14.237251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 79/154] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:14 2022] GET /identity/v3/users/8dcd62677ffc4535814a24f589fce966 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:14.242968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63526b6-57eb-4dff-8fb3-c90fd2552f52 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:14.364481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:14.396772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63526b6-57eb-4dff-8fb3-c90fd2552f52 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:14.442451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e6e09-3d29-4da2-a363-b415df3c7cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:14.443170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e6e09-3d29-4da2-a363-b415df3c7cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:14.443406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e6e09-3d29-4da2-a363-b415df3c7cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:14.443689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e6e09-3d29-4da2-a363-b415df3c7cf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:14.525476 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e6e09-3d29-4da2-a363-b415df3c7cf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:14.526970 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e6e09-3d29-4da2-a363-b415df3c7cf7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:14.616475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:14.645654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 76/155] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:14.654771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79561924-b831-4274-b162-c466e7b75481 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:14.748523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:14.761532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf3f3f-135c-439a-bf9f-23b3ead053e3 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:14.761856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf3f3f-135c-439a-bf9f-23b3ead053e3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:14.762107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf3f3f-135c-439a-bf9f-23b3ead053e3 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:14.763560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 77/156] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:14.772127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e50bc-e30e-4f4b-8291-1289a544083d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:14.772424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e50bc-e30e-4f4b-8291-1289a544083d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:14.772697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e50bc-e30e-4f4b-8291-1289a544083d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:14.800747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79561924-b831-4274-b162-c466e7b75481 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:14.802196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-834e50bc-e30e-4f4b-8291-1289a544083d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:14.803427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-834e50bc-e30e-4f4b-8291-1289a544083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:14.856167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c950b54-0ec7-4d94-af3e-f1a7ec5df167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:14.857595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c950b54-0ec7-4d94-af3e-f1a7ec5df167 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:14.857868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c950b54-0ec7-4d94-af3e-f1a7ec5df167 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:14.858139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c950b54-0ec7-4d94-af3e-f1a7ec5df167 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:14.859381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c950b54-0ec7-4d94-af3e-f1a7ec5df167 None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:14.860968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c950b54-0ec7-4d94-af3e-f1a7ec5df167 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:14.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 80/157] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:14 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:14.870944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c950b54-0ec7-4d94-af3e-f1a7ec5df167 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:14.900377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 78/158] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:14.947483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e50bc-e30e-4f4b-8291-1289a544083d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:15.110992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c950b54-0ec7-4d94-af3e-f1a7ec5df167 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:15.181358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e50bc-e30e-4f4b-8291-1289a544083d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:15.197685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:15.202793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:15.213845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc58d-fe57-43b3-86b5-649e4c40a754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:15.214835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc58d-fe57-43b3-86b5-649e4c40a754 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:15.214835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc58d-fe57-43b3-86b5-649e4c40a754 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:15.215112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc58d-fe57-43b3-86b5-649e4c40a754 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:15.216545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddc58d-fe57-43b3-86b5-649e4c40a754 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=26049e1a44a94f1ab013063cd9767d44, user.id=8dcd62677ffc4535814a24f589fce966) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:15.216795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddc58d-fe57-43b3-86b5-649e4c40a754 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:15.218240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddc58d-fe57-43b3-86b5-649e4c40a754 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:15.224587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 81/159] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:14 2022] GET /identity/v3/role_assignments?user.id=8dcd62677ffc4535814a24f589fce966&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=e089a0d6040040c4bcf143a680488e3d => generated 608 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:15.269338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ca873-854c-47ff-9df0-bda889c32824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:15.270206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ca873-854c-47ff-9df0-bda889c32824 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:15.270492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ca873-854c-47ff-9df0-bda889c32824 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:15.270769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ca873-854c-47ff-9df0-bda889c32824 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:15.298133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ca873-854c-47ff-9df0-bda889c32824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-e46ca873-854c-47ff-9df0-bda889c32824 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:15.306284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:15.307911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:15.310096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:15.312988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 79/160] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:15.314557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc58d-fe57-43b3-86b5-649e4c40a754 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:15.480539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc58d-fe57-43b3-86b5-649e4c40a754 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:15.516867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bef1f1-4046-4632-9371-138b29a06a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:15.517573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bef1f1-4046-4632-9371-138b29a06a50 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:15.517872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bef1f1-4046-4632-9371-138b29a06a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:15.518119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bef1f1-4046-4632-9371-138b29a06a50 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:15.518874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bef1f1-4046-4632-9371-138b29a06a50 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:15.519181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bef1f1-4046-4632-9371-138b29a06a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:15.520560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bef1f1-4046-4632-9371-138b29a06a50 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:15.530591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 82/161] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:15.536042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ca873-854c-47ff-9df0-bda889c32824 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:15.727372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a1750-a505-4c4f-aba8-2be40544c58e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:15.727633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a1750-a505-4c4f-aba8-2be40544c58e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:15.727963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a1750-a505-4c4f-aba8-2be40544c58e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:15.729610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 83/162] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:15.738531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732a505-39ce-4262-8b90-1c4908997b9e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:15.738722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732a505-39ce-4262-8b90-1c4908997b9e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:15.739009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732a505-39ce-4262-8b90-1c4908997b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:15.749631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ca873-854c-47ff-9df0-bda889c32824 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:15.782132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2732a505-39ce-4262-8b90-1c4908997b9e None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:15.783339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2732a505-39ce-4262-8b90-1c4908997b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:15.827114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c43ab-e5d8-48d4-9fe6-097f26a01c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:15.829477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c43ab-e5d8-48d4-9fe6-097f26a01c5e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:15.831752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c43ab-e5d8-48d4-9fe6-097f26a01c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:15.832735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c43ab-e5d8-48d4-9fe6-097f26a01c5e None admin] PATH_INFO: `/v3/users/a18753029f6c4adaa1a4d5f579d67d55` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:15.837124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c43ab-e5d8-48d4-9fe6-097f26a01c5e None admin] RBAC: Authorizing `identity:get_user(user_id=a18753029f6c4adaa1a4d5f579d67d55)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:15.840228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c43ab-e5d8-48d4-9fe6-097f26a01c5e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:15.849796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 80/163] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:15 2022] GET /identity/v3/users/a18753029f6c4adaa1a4d5f579d67d55 => generated 289 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:15.855273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c43ab-e5d8-48d4-9fe6-097f26a01c5e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:15.953866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:15.966948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:16.005549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 84/164] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:16.007455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e0796-f946-470a-ab21-fabba1321629 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:16.007940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e0796-f946-470a-ab21-fabba1321629 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:16.008415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e0796-f946-470a-ab21-fabba1321629 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:16.015624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 85/165] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:16.021821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30696e5-fc0c-4795-9309-db589b3e7e66 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:16.022303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30696e5-fc0c-4795-9309-db589b3e7e66 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:16.022786 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30696e5-fc0c-4795-9309-db589b3e7e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:16.068086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e30696e5-fc0c-4795-9309-db589b3e7e66 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:16.069754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e30696e5-fc0c-4795-9309-db589b3e7e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:16.121893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c43ab-e5d8-48d4-9fe6-097f26a01c5e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:16.166809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e18976-9149-4292-8fd6-3204c14b7fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:16.168086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e18976-9149-4292-8fd6-3204c14b7fec None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:16.168604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e18976-9149-4292-8fd6-3204c14b7fec None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:16.171092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e18976-9149-4292-8fd6-3204c14b7fec None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:16.173006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e18976-9149-4292-8fd6-3204c14b7fec None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:16.174766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e18976-9149-4292-8fd6-3204c14b7fec None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:16.177140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 81/166] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:15 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:16.178228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e18976-9149-4292-8fd6-3204c14b7fec None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:16.240006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 86/167] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:16.240546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30696e5-fc0c-4795-9309-db589b3e7e66 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:16.292784 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e18976-9149-4292-8fd6-3204c14b7fec None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:16.296033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:16.344534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841bae4-fa03-4e5e-86f6-cb47f9aad17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:16.345653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841bae4-fa03-4e5e-86f6-cb47f9aad17f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:16.346097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841bae4-fa03-4e5e-86f6-cb47f9aad17f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:16.346497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841bae4-fa03-4e5e-86f6-cb47f9aad17f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:16.358664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841bae4-fa03-4e5e-86f6-cb47f9aad17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2841bae4-fa03-4e5e-86f6-cb47f9aad17f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:16.367541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:16.368841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:16.369824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:16.372067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 82/168] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:16.373620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841bae4-fa03-4e5e-86f6-cb47f9aad17f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:16.416841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30696e5-fc0c-4795-9309-db589b3e7e66 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:16.449754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:16.487443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5c181-0529-4531-8942-f4bcd8cbae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:16.488578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5c181-0529-4531-8942-f4bcd8cbae84 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:16.489032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5c181-0529-4531-8942-f4bcd8cbae84 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:16.489476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5c181-0529-4531-8942-f4bcd8cbae84 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44/users/a18753029f6c4adaa1a4d5f579d67d55/roles/f216586d507240c9b31a20c99b9bd21a` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:16.491694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5c181-0529-4531-8942-f4bcd8cbae84 None admin] RBAC: Authorizing `identity:create_grant(project_id=26049e1a44a94f1ab013063cd9767d44, user_id=a18753029f6c4adaa1a4d5f579d67d55, role_id=f216586d507240c9b31a20c99b9bd21a)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:16.493397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5c181-0529-4531-8942-f4bcd8cbae84 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:16.509441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 87/169] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:49:16 2022] PUT /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44/users/a18753029f6c4adaa1a4d5f579d67d55/roles/f216586d507240c9b31a20c99b9bd21a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:16.510678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5c181-0529-4531-8942-f4bcd8cbae84 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:16.579343 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841bae4-fa03-4e5e-86f6-cb47f9aad17f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:16.644467 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5c181-0529-4531-8942-f4bcd8cbae84 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:16.679096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e1711-9a4b-45ef-ac83-7f5a756da811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:16.679886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e1711-9a4b-45ef-ac83-7f5a756da811 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:16.680181 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e1711-9a4b-45ef-ac83-7f5a756da811 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:16.680469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e1711-9a4b-45ef-ac83-7f5a756da811 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:16.688029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c539-7428-4147-acbc-16ec491d85b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:16.689409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c539-7428-4147-acbc-16ec491d85b3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:16.689856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c539-7428-4147-acbc-16ec491d85b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:16.690328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c539-7428-4147-acbc-16ec491d85b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:16.691383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3c539-7428-4147-acbc-16ec491d85b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:16.691891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3c539-7428-4147-acbc-16ec491d85b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:16.693570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3c539-7428-4147-acbc-16ec491d85b3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:16.702743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e1711-9a4b-45ef-ac83-7f5a756da811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:16.704129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 88/170] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:16.709621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c539-7428-4147-acbc-16ec491d85b3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-f13e1711-9a4b-45ef-ac83-7f5a756da811 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:16.714935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:16.716360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:16.717173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 83/171] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:16.726239 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e1711-9a4b-45ef-ac83-7f5a756da811 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:16.760716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:16.790419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:16.837490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c539-7428-4147-acbc-16ec491d85b3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:16.876384 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207723b5-6235-4864-80b8-d629afa0c6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:16.877534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207723b5-6235-4864-80b8-d629afa0c6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:16.877989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207723b5-6235-4864-80b8-d629afa0c6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:16.878409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207723b5-6235-4864-80b8-d629afa0c6e7 None admin] PATH_INFO: `/v3/users/d809509539f742ddb1d244a1c6f9e2af` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:16.879791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207723b5-6235-4864-80b8-d629afa0c6e7 None admin] RBAC: Authorizing `identity:get_user(user_id=d809509539f742ddb1d244a1c6f9e2af)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:16.881659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207723b5-6235-4864-80b8-d629afa0c6e7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:16.883805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 89/172] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:16 2022] GET /identity/v3/users/d809509539f742ddb1d244a1c6f9e2af => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:16.890898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207723b5-6235-4864-80b8-d629afa0c6e7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:16.987297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e1711-9a4b-45ef-ac83-7f5a756da811 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:17.019400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207723b5-6235-4864-80b8-d629afa0c6e7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:17.026257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f03d4c-822a-4f92-96a3-8f123cbf8a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:17.026938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f03d4c-822a-4f92-96a3-8f123cbf8a11 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.027229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f03d4c-822a-4f92-96a3-8f123cbf8a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.027511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f03d4c-822a-4f92-96a3-8f123cbf8a11 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.028937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f03d4c-822a-4f92-96a3-8f123cbf8a11 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:17.029284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f03d4c-822a-4f92-96a3-8f123cbf8a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:17.030503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:17.030705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f03d4c-822a-4f92-96a3-8f123cbf8a11 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:17.040888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 84/173] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:17.045678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24368e-03a2-4dd9-8228-932337eb70aa None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.045977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24368e-03a2-4dd9-8228-932337eb70aa None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.046264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24368e-03a2-4dd9-8228-932337eb70aa None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.051717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 85/174] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:17.054532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24368e-03a2-4dd9-8228-932337eb70aa None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:17.065040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5c258-c242-48e0-8dbb-7862f09a8874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:17.066187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5c258-c242-48e0-8dbb-7862f09a8874 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.067161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5c258-c242-48e0-8dbb-7862f09a8874 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.067615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5c258-c242-48e0-8dbb-7862f09a8874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.156595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5c258-c242-48e0-8dbb-7862f09a8874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:17.158499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5c258-c242-48e0-8dbb-7862f09a8874 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:17.232399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:17.266966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24368e-03a2-4dd9-8228-932337eb70aa None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:17.273779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 90/175] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:17.275919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0dcab-7b4f-4c2d-af7d-81d352eb6e8b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.276502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0dcab-7b4f-4c2d-af7d-81d352eb6e8b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.276990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0dcab-7b4f-4c2d-af7d-81d352eb6e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.288954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:17.295773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9dd0dcab-7b4f-4c2d-af7d-81d352eb6e8b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:17.297099 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9dd0dcab-7b4f-4c2d-af7d-81d352eb6e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:17.342432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f346d-5ef4-498a-b850-fbab1348ed6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:17.343316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f346d-5ef4-498a-b850-fbab1348ed6f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.343598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f346d-5ef4-498a-b850-fbab1348ed6f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.343872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f346d-5ef4-498a-b850-fbab1348ed6f None admin] PATH_INFO: `/v3/users/d809509539f742ddb1d244a1c6f9e2af` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.349923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f346d-5ef4-498a-b850-fbab1348ed6f None admin] RBAC: Authorizing `identity:get_user(user_id=d809509539f742ddb1d244a1c6f9e2af)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:17.351409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f346d-5ef4-498a-b850-fbab1348ed6f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:17.353485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 86/176] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:17 2022] GET /identity/v3/users/d809509539f742ddb1d244a1c6f9e2af => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:17.354286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f346d-5ef4-498a-b850-fbab1348ed6f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:17.381005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 91/177] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:17.381704 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0dcab-7b4f-4c2d-af7d-81d352eb6e8b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:17.486698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f346d-5ef4-498a-b850-fbab1348ed6f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:17.488912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0dcab-7b4f-4c2d-af7d-81d352eb6e8b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:17.521599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf26b23-52d4-4080-bbbc-dc4b298e2a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:17.522462 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf26b23-52d4-4080-bbbc-dc4b298e2a53 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.522749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf26b23-52d4-4080-bbbc-dc4b298e2a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.523026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf26b23-52d4-4080-bbbc-dc4b298e2a53 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.524193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf26b23-52d4-4080-bbbc-dc4b298e2a53 None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:17.525049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445532f-e05a-4467-a41b-c518d3a78e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:17.525675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf26b23-52d4-4080-bbbc-dc4b298e2a53 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:17.525770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445532f-e05a-4467-a41b-c518d3a78e42 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.526002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445532f-e05a-4467-a41b-c518d3a78e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.526279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445532f-e05a-4467-a41b-c518d3a78e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.527373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 87/178] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:17 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:17.529080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4d6a-944e-434f-b426-c8ef841c0d6e None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.529386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4d6a-944e-434f-b426-c8ef841c0d6e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.530311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4d6a-944e-434f-b426-c8ef841c0d6e None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.530917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:17.535073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 88/179] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:17.536091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4d6a-944e-434f-b426-c8ef841c0d6e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:17.562240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:17.603107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445532f-e05a-4467-a41b-c518d3a78e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:17.604583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445532f-e05a-4467-a41b-c518d3a78e42 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:17.650085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4d6a-944e-434f-b426-c8ef841c0d6e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:17.685827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a948c3-2bb6-41fe-9791-1178d972fb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:17.686662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a948c3-2bb6-41fe-9791-1178d972fb54 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.686906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a948c3-2bb6-41fe-9791-1178d972fb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.687188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a948c3-2bb6-41fe-9791-1178d972fb54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.698637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a948c3-2bb6-41fe-9791-1178d972fb54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-36a948c3-2bb6-41fe-9791-1178d972fb54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:17.706935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:17.707955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:17.708901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 89/180] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:17.709731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be935b95-9e46-48e3-950d-8e296576944f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.709731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be935b95-9e46-48e3-950d-8e296576944f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.710020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be935b95-9e46-48e3-950d-8e296576944f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.711289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 90/181] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:17.715386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ec66a-3ffc-4a6d-9e0b-d6429056f916 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.715386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ec66a-3ffc-4a6d-9e0b-d6429056f916 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.715386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ec66a-3ffc-4a6d-9e0b-d6429056f916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.721391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 92/182] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:17.726761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445532f-e05a-4467-a41b-c518d3a78e42 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:17.733198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7c0ec66a-3ffc-4a6d-9e0b-d6429056f916 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:17.734517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7c0ec66a-3ffc-4a6d-9e0b-d6429056f916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:17.765116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:17.817520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 91/183] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:17.818285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ec66a-3ffc-4a6d-9e0b-d6429056f916 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:17.836571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:17.837025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445532f-e05a-4467-a41b-c518d3a78e42 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:17.875457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefc8df-9d4e-437f-ad0b-7e0d40895e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:17.876119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefc8df-9d4e-437f-ad0b-7e0d40895e19 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.876408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefc8df-9d4e-437f-ad0b-7e0d40895e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.876689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefc8df-9d4e-437f-ad0b-7e0d40895e19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.877564 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefc8df-9d4e-437f-ad0b-7e0d40895e19 None admin] RBAC: Adding query filter params (role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=26049e1a44a94f1ab013063cd9767d44, user.id=d809509539f742ddb1d244a1c6f9e2af) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:17.877885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefc8df-9d4e-437f-ad0b-7e0d40895e19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:17.879335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefc8df-9d4e-437f-ad0b-7e0d40895e19 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:17.885577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 93/184] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:17 2022] GET /identity/v3/role_assignments?user.id=d809509539f742ddb1d244a1c6f9e2af&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=8184510545604582b267e4d1871ae9bf => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:17.898663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743db146-7d70-4c4b-8966-862084120083 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.898986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743db146-7d70-4c4b-8966-862084120083 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.899269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743db146-7d70-4c4b-8966-862084120083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.922454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-743db146-7d70-4c4b-8966-862084120083 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:17.923684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-743db146-7d70-4c4b-8966-862084120083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:17.942840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ec66a-3ffc-4a6d-9e0b-d6429056f916 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:17.994506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c2c0d-7008-4d2e-a791-9d1f811f10db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:17.995198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c2c0d-7008-4d2e-a791-9d1f811f10db None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:17.995483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c2c0d-7008-4d2e-a791-9d1f811f10db None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:17.995759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c2c0d-7008-4d2e-a791-9d1f811f10db None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:17.996533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c2c0d-7008-4d2e-a791-9d1f811f10db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:17.997383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c2c0d-7008-4d2e-a791-9d1f811f10db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:17.998826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c2c0d-7008-4d2e-a791-9d1f811f10db None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:18.006829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 94/185] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:18.007770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743db146-7d70-4c4b-8966-862084120083 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.015289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 92/186] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:18.016104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c2c0d-7008-4d2e-a791-9d1f811f10db None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.042936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:18.125139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743db146-7d70-4c4b-8966-862084120083 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:18.126472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:18.158544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c2c0d-7008-4d2e-a791-9d1f811f10db None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:18.175110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191473e-4282-49f9-bf64-41fc7ce090f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:18.176516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191473e-4282-49f9-bf64-41fc7ce090f9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:18.176983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191473e-4282-49f9-bf64-41fc7ce090f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:18.177644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191473e-4282-49f9-bf64-41fc7ce090f9 None admin] PATH_INFO: `/v3/projects/5de4e47053624c3ebc24ad72778c18ac` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:18.179213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191473e-4282-49f9-bf64-41fc7ce090f9 None admin] RBAC: Authorizing `identity:get_project(project_id=5de4e47053624c3ebc24ad72778c18ac)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:18.180676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191473e-4282-49f9-bf64-41fc7ce090f9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:18.183090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 95/187] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/projects/5de4e47053624c3ebc24ad72778c18ac => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:18.184026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191473e-4282-49f9-bf64-41fc7ce090f9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.198358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6a98-6d04-4f57-a1e5-45fa51efc13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:18.199172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6a98-6d04-4f57-a1e5-45fa51efc13b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:18.199485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6a98-6d04-4f57-a1e5-45fa51efc13b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:18.199739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6a98-6d04-4f57-a1e5-45fa51efc13b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:18.211816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b6a98-6d04-4f57-a1e5-45fa51efc13b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-331b6a98-6d04-4f57-a1e5-45fa51efc13b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:18.220354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:18.221419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:18.222470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:18.224491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 93/188] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:18.225435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6a98-6d04-4f57-a1e5-45fa51efc13b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.296009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:18.301248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191473e-4282-49f9-bf64-41fc7ce090f9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:18.338699 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6a98-6d04-4f57-a1e5-45fa51efc13b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:18.344374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36f9e2-22ce-4813-a838-1cd9d4e20d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:18.345298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36f9e2-22ce-4813-a838-1cd9d4e20d07 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:18.345589 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36f9e2-22ce-4813-a838-1cd9d4e20d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:18.345872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36f9e2-22ce-4813-a838-1cd9d4e20d07 None admin] PATH_INFO: `/v3/users/0b02aef84a6345038b983f7e2c63cdee` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:18.361345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36f9e2-22ce-4813-a838-1cd9d4e20d07 None admin] RBAC: Authorizing `identity:get_user(user_id=0b02aef84a6345038b983f7e2c63cdee)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:18.362826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36f9e2-22ce-4813-a838-1cd9d4e20d07 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:18.364729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 96/189] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/users/0b02aef84a6345038b983f7e2c63cdee => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:18.366605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36f9e2-22ce-4813-a838-1cd9d4e20d07 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.375213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd37c3f-49bf-4dac-a53d-8cbfcbc4a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:18.376014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd37c3f-49bf-4dac-a53d-8cbfcbc4a4ef None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:18.376301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd37c3f-49bf-4dac-a53d-8cbfcbc4a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:18.376582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd37c3f-49bf-4dac-a53d-8cbfcbc4a4ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:18.378136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd37c3f-49bf-4dac-a53d-8cbfcbc4a4ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:18.379260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd37c3f-49bf-4dac-a53d-8cbfcbc4a4ef None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:18.380897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 94/190] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:18.381697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd37c3f-49bf-4dac-a53d-8cbfcbc4a4ef None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.388095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:18.477258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36f9e2-22ce-4813-a838-1cd9d4e20d07 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:18.490601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd37c3f-49bf-4dac-a53d-8cbfcbc4a4ef None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:18.515187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5594784-a556-423f-a318-7ac869e066ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:18.516173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5594784-a556-423f-a318-7ac869e066ed None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:18.516635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5594784-a556-423f-a318-7ac869e066ed None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:18.517203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5594784-a556-423f-a318-7ac869e066ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:18.518357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5594784-a556-423f-a318-7ac869e066ed None admin] RBAC: Adding query filter params (role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=5de4e47053624c3ebc24ad72778c18ac, user.id=d809509539f742ddb1d244a1c6f9e2af) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:18.518856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5594784-a556-423f-a318-7ac869e066ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:18.520701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5594784-a556-423f-a318-7ac869e066ed None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:18.525246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c08d9-6907-45f4-bea2-ee572258a430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:18.525922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c08d9-6907-45f4-bea2-ee572258a430 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:18.526210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c08d9-6907-45f4-bea2-ee572258a430 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:18.526491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c08d9-6907-45f4-bea2-ee572258a430 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:18.527257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c08d9-6907-45f4-bea2-ee572258a430 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:18.527569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c08d9-6907-45f4-bea2-ee572258a430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:18.528451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 97/191] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/role_assignments?user.id=d809509539f742ddb1d244a1c6f9e2af&scope.project.id=5de4e47053624c3ebc24ad72778c18ac&role.id=301682fc5c92440f874912eabfed6f70 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:18.528992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c08d9-6907-45f4-bea2-ee572258a430 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:18.529650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5594784-a556-423f-a318-7ac869e066ed None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.546766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 95/192] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:18.548002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c08d9-6907-45f4-bea2-ee572258a430 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.566757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:18.625298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:18.663669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5594784-a556-423f-a318-7ac869e066ed None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:18.672801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c08d9-6907-45f4-bea2-ee572258a430 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:18.704854 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f199ad-957f-4e01-8b83-a64873e39d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:18.705714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f199ad-957f-4e01-8b83-a64873e39d25 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:18.706101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f199ad-957f-4e01-8b83-a64873e39d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:18.706634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f199ad-957f-4e01-8b83-a64873e39d25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:18.718851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993f801-9e17-40da-b217-ba4ae730ce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:18.719532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f801-9e17-40da-b217-ba4ae730ce0f None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:18.719817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f801-9e17-40da-b217-ba4ae730ce0f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:18.720581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f801-9e17-40da-b217-ba4ae730ce0f None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:18.721907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993f801-9e17-40da-b217-ba4ae730ce0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:18.724808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993f801-9e17-40da-b217-ba4ae730ce0f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:18.748969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 96/193] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:18 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:18.750427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993f801-9e17-40da-b217-ba4ae730ce0f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.804609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f199ad-957f-4e01-8b83-a64873e39d25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:18.806162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f199ad-957f-4e01-8b83-a64873e39d25 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:18.833642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:18.905553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993f801-9e17-40da-b217-ba4ae730ce0f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:18.932007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 98/194] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:18.932967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:18.938452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f199ad-957f-4e01-8b83-a64873e39d25 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:18.946194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b05d41-aad0-4d22-88b8-7ecb83a27149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:18.947076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b05d41-aad0-4d22-88b8-7ecb83a27149 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:18.947359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b05d41-aad0-4d22-88b8-7ecb83a27149 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:18.947639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b05d41-aad0-4d22-88b8-7ecb83a27149 None admin] PATH_INFO: `/v3/users/0b02aef84a6345038b983f7e2c63cdee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:18.948867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b05d41-aad0-4d22-88b8-7ecb83a27149 None admin] RBAC: Authorizing `identity:get_user(user_id=0b02aef84a6345038b983f7e2c63cdee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:18.950428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b05d41-aad0-4d22-88b8-7ecb83a27149 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:18.952145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 97/195] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/users/0b02aef84a6345038b983f7e2c63cdee => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:18.957547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b05d41-aad0-4d22-88b8-7ecb83a27149 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:19.058984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f199ad-957f-4e01-8b83-a64873e39d25 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:19.084053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b05d41-aad0-4d22-88b8-7ecb83a27149 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:19.088061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:19.110519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24bc5c-6864-4d23-b5ea-af8f378ea51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:19.111764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24bc5c-6864-4d23-b5ea-af8f378ea51a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:19.112197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24bc5c-6864-4d23-b5ea-af8f378ea51a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:19.112623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24bc5c-6864-4d23-b5ea-af8f378ea51a None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:19.115297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24bc5c-6864-4d23-b5ea-af8f378ea51a None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:19.116915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24bc5c-6864-4d23-b5ea-af8f378ea51a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:19.121197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 99/196] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:19.122772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665aec-ead5-4911-bff3-49d1a2777c57 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:19.123062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665aec-ead5-4911-bff3-49d1a2777c57 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:19.123371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665aec-ead5-4911-bff3-49d1a2777c57 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:19.124678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 100/197] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:19.127574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08d669-296f-468f-89f4-8129e9ec368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:19.128454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08d669-296f-468f-89f4-8129e9ec368e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:19.128908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08d669-296f-468f-89f4-8129e9ec368e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:19.129435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08d669-296f-468f-89f4-8129e9ec368e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:19.133136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d6e0d-e3d0-4ed5-b91c-f4c2350ca888 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:19.133434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d6e0d-e3d0-4ed5-b91c-f4c2350ca888 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:19.133719 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d6e0d-e3d0-4ed5-b91c-f4c2350ca888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:19.155141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a41d6e0d-e3d0-4ed5-b91c-f4c2350ca888 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:19.156455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a41d6e0d-e3d0-4ed5-b91c-f4c2350ca888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:19.201658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:19.214847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08d669-296f-468f-89f4-8129e9ec368e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:19.216931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08d669-296f-468f-89f4-8129e9ec368e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:19.241290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 101/198] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:19.242561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d6e0d-e3d0-4ed5-b91c-f4c2350ca888 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:19.328607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 98/199] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:19.333815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08d669-296f-468f-89f4-8129e9ec368e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:19.388253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d6e0d-e3d0-4ed5-b91c-f4c2350ca888 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:19.414820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:19.432685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:19.443997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b5f96-61c4-4cfc-b71e-3aafa2bb3acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:19.445164 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b5f96-61c4-4cfc-b71e-3aafa2bb3acf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:19.445594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b5f96-61c4-4cfc-b71e-3aafa2bb3acf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:19.446380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b5f96-61c4-4cfc-b71e-3aafa2bb3acf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:19.447646 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b5f96-61c4-4cfc-b71e-3aafa2bb3acf None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=45204248d1df42a29cab699e5a889efe, user.id=0b02aef84a6345038b983f7e2c63cdee) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:19.448163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b5f96-61c4-4cfc-b71e-3aafa2bb3acf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:19.448886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08d669-296f-468f-89f4-8129e9ec368e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:19.450007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b5f96-61c4-4cfc-b71e-3aafa2bb3acf None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:19.458876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 102/200] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:19 2022] GET /identity/v3/role_assignments?user.id=0b02aef84a6345038b983f7e2c63cdee&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=e089a0d6040040c4bcf143a680488e3d => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:19.459898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b5f96-61c4-4cfc-b71e-3aafa2bb3acf None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:19.525368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac666f-6e16-410d-b02a-1ccd069ed6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:19.529433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac666f-6e16-410d-b02a-1ccd069ed6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:19.529936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac666f-6e16-410d-b02a-1ccd069ed6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:19.530363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac666f-6e16-410d-b02a-1ccd069ed6b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:19.552564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac666f-6e16-410d-b02a-1ccd069ed6b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-44ac666f-6e16-410d-b02a-1ccd069ed6b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:19.561999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:19.563383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:19.564369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:19.566597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 99/201] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:19.574161 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac666f-6e16-410d-b02a-1ccd069ed6b6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:19.636002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b5f96-61c4-4cfc-b71e-3aafa2bb3acf None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:19.707107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788d703-64bc-40ae-aad2-22b61e58dbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:19.710521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788d703-64bc-40ae-aad2-22b61e58dbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:19.711175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788d703-64bc-40ae-aad2-22b61e58dbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:19.711693 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788d703-64bc-40ae-aad2-22b61e58dbc7 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:19.714314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788d703-64bc-40ae-aad2-22b61e58dbc7 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:19.716302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788d703-64bc-40ae-aad2-22b61e58dbc7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:19.719382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 103/202] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:19 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:19.725334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788d703-64bc-40ae-aad2-22b61e58dbc7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:19.780342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac666f-6e16-410d-b02a-1ccd069ed6b6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:19.822813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:19.858653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05007f-4e53-4f0c-9d9d-2e0dab276c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:19.859708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05007f-4e53-4f0c-9d9d-2e0dab276c49 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:19.860192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05007f-4e53-4f0c-9d9d-2e0dab276c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:19.860666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05007f-4e53-4f0c-9d9d-2e0dab276c49 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:19.861652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05007f-4e53-4f0c-9d9d-2e0dab276c49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:19.862185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05007f-4e53-4f0c-9d9d-2e0dab276c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:19.863948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05007f-4e53-4f0c-9d9d-2e0dab276c49 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:19.883454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 100/203] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:19.887863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05007f-4e53-4f0c-9d9d-2e0dab276c49 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:19.912106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:19.920620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788d703-64bc-40ae-aad2-22b61e58dbc7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:19.989697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c32381-3508-4d3b-ac40-0a114adcade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:19.990795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c32381-3508-4d3b-ac40-0a114adcade2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:19.991417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c32381-3508-4d3b-ac40-0a114adcade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:19.992599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c32381-3508-4d3b-ac40-0a114adcade2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:19.993682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c32381-3508-4d3b-ac40-0a114adcade2 None admin] RBAC: Adding query filter params (role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=45204248d1df42a29cab699e5a889efe, user.id=0b02aef84a6345038b983f7e2c63cdee) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:19.994393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c32381-3508-4d3b-ac40-0a114adcade2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:19.999307 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c32381-3508-4d3b-ac40-0a114adcade2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:20.008577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 104/204] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:19 2022] GET /identity/v3/role_assignments?user.id=0b02aef84a6345038b983f7e2c63cdee&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=8184510545604582b267e4d1871ae9bf => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:20.090818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05007f-4e53-4f0c-9d9d-2e0dab276c49 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:20.152923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e6a0d-64d7-4d49-8a69-f380dbc5869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:20.154759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e6a0d-64d7-4d49-8a69-f380dbc5869a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:20.155383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e6a0d-64d7-4d49-8a69-f380dbc5869a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:20.155862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e6a0d-64d7-4d49-8a69-f380dbc5869a None admin] PATH_INFO: `/v3/users/a18753029f6c4adaa1a4d5f579d67d55` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:20.157435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e6a0d-64d7-4d49-8a69-f380dbc5869a None admin] RBAC: Authorizing `identity:get_user(user_id=a18753029f6c4adaa1a4d5f579d67d55)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:20.159154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e6a0d-64d7-4d49-8a69-f380dbc5869a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:20.161074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 101/205] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:19 2022] GET /identity/v3/users/a18753029f6c4adaa1a4d5f579d67d55 => generated 289 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:20.167059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c32381-3508-4d3b-ac40-0a114adcade2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:20.287913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c32381-3508-4d3b-ac40-0a114adcade2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:20.303212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:20.335437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb970b13-44ff-4337-9fc7-21c980543ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:20.336479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb970b13-44ff-4337-9fc7-21c980543ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:20.336966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb970b13-44ff-4337-9fc7-21c980543ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:20.337468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb970b13-44ff-4337-9fc7-21c980543ef5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:20.420955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb970b13-44ff-4337-9fc7-21c980543ef5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:20.423186 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb970b13-44ff-4337-9fc7-21c980543ef5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:20.536458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 105/206] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:20.542256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e6a0d-64d7-4d49-8a69-f380dbc5869a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:20.639302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:20.673670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e6a0d-64d7-4d49-8a69-f380dbc5869a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:20.712482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367a13d-0685-464c-a1c5-bea12a7bc093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:20.713788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a13d-0685-464c-a1c5-bea12a7bc093 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:20.714304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a13d-0685-464c-a1c5-bea12a7bc093 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:20.714778 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a13d-0685-464c-a1c5-bea12a7bc093 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:20.716736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367a13d-0685-464c-a1c5-bea12a7bc093 None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:20.718493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367a13d-0685-464c-a1c5-bea12a7bc093 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:20.720813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 102/207] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:20 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:20.726814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb970b13-44ff-4337-9fc7-21c980543ef5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:20.746302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:20.857741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb970b13-44ff-4337-9fc7-21c980543ef5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:20.908610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0843-9cd8-47f9-87d4-3e34b5b9d056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:20.909297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0843-9cd8-47f9-87d4-3e34b5b9d056 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:20.909585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0843-9cd8-47f9-87d4-3e34b5b9d056 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:20.909869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0843-9cd8-47f9-87d4-3e34b5b9d056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:20.910748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f0843-9cd8-47f9-87d4-3e34b5b9d056 None admin] RBAC: Adding query filter params (role.id=f216586d507240c9b31a20c99b9bd21a, scope.project.id=26049e1a44a94f1ab013063cd9767d44, user.id=a18753029f6c4adaa1a4d5f579d67d55) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:20.911075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f0843-9cd8-47f9-87d4-3e34b5b9d056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:20.912534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f0843-9cd8-47f9-87d4-3e34b5b9d056 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:20.925365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 106/208] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:20 2022] GET /identity/v3/role_assignments?user.id=a18753029f6c4adaa1a4d5f579d67d55&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=f216586d507240c9b31a20c99b9bd21a => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:21.409923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be597b61-6e47-48a6-8589-1df889f25c2f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.411025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be597b61-6e47-48a6-8589-1df889f25c2f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.411309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be597b61-6e47-48a6-8589-1df889f25c2f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:21.413142 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 103/209] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:21.418745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bc424-8491-4188-9aa4-e0d72914921a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.419025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bc424-8491-4188-9aa4-e0d72914921a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.419294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bc424-8491-4188-9aa4-e0d72914921a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:21.438848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f41bc424-8491-4188-9aa4-e0d72914921a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:21.440062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f41bc424-8491-4188-9aa4-e0d72914921a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:21.462029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa7934-afe0-4413-8191-bc089ab7342e None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.462352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa7934-afe0-4413-8191-bc089ab7342e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.462612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa7934-afe0-4413-8191-bc089ab7342e None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:21.464251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 104/210] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:21.471035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e74cf-671a-495b-9a68-6ee990de37d3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.471335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e74cf-671a-495b-9a68-6ee990de37d3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.471616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e74cf-671a-495b-9a68-6ee990de37d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:21.496404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-bd5e74cf-671a-495b-9a68-6ee990de37d3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:21.497666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bd5e74cf-671a-495b-9a68-6ee990de37d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:21.530393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 107/211] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:21.553899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bc424-8491-4188-9aa4-e0d72914921a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:21.564473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:21.586373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 105/212] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:21.611189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e74cf-671a-495b-9a68-6ee990de37d3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:21.663740 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bc424-8491-4188-9aa4-e0d72914921a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:21.692610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:21.699209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d43f51-544d-4e5e-b53c-ab3591d44b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:21.700720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d43f51-544d-4e5e-b53c-ab3591d44b91 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.700929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d43f51-544d-4e5e-b53c-ab3591d44b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.701192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d43f51-544d-4e5e-b53c-ab3591d44b91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:21.712591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d43f51-544d-4e5e-b53c-ab3591d44b91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:21.720539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e74cf-671a-495b-9a68-6ee990de37d3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-80d43f51-544d-4e5e-b53c-ab3591d44b91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:21.720805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:21.721872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:21.723039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 108/213] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:21.727800 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d43f51-544d-4e5e-b53c-ab3591d44b91 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:21.761389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da985bac-2698-4c9f-ad6f-8d88ef767bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:21.763742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da985bac-2698-4c9f-ad6f-8d88ef767bae None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.764229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da985bac-2698-4c9f-ad6f-8d88ef767bae None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.764684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da985bac-2698-4c9f-ad6f-8d88ef767bae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:21.766362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da985bac-2698-4c9f-ad6f-8d88ef767bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:21.768909 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da985bac-2698-4c9f-ad6f-8d88ef767bae None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:21.770792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 106/214] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:21.776268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da985bac-2698-4c9f-ad6f-8d88ef767bae None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:21.811622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:21.852412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d43f51-544d-4e5e-b53c-ab3591d44b91 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:21.891109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e5881-8d6a-40a0-8cf8-d11a7b888cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:21.891888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e5881-8d6a-40a0-8cf8-d11a7b888cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.892181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e5881-8d6a-40a0-8cf8-d11a7b888cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.892589 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e5881-8d6a-40a0-8cf8-d11a7b888cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:21.893294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e5881-8d6a-40a0-8cf8-d11a7b888cb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:21.893606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e5881-8d6a-40a0-8cf8-d11a7b888cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:21.895097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e5881-8d6a-40a0-8cf8-d11a7b888cb2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:21.901422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da985bac-2698-4c9f-ad6f-8d88ef767bae None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:21.905561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 109/215] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:21.911050 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e5881-8d6a-40a0-8cf8-d11a7b888cb2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:21.939935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce75d5-d058-4aee-8330-7205905fca3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:21.940988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce75d5-d058-4aee-8330-7205905fca3e None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.941501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce75d5-d058-4aee-8330-7205905fca3e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.941991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce75d5-d058-4aee-8330-7205905fca3e None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:21.942918 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce75d5-d058-4aee-8330-7205905fca3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:21.944561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce75d5-d058-4aee-8330-7205905fca3e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:21.964540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:21.977259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 107/216] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:21 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:49:21.977259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00e895-7160-47e1-9040-9eaf2d906bea None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.977259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00e895-7160-47e1-9040-9eaf2d906bea None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.977259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00e895-7160-47e1-9040-9eaf2d906bea None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:21.984534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 108/217] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:21.986219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a77f0-8385-462e-9360-0a48161d7245 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:21.986624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a77f0-8385-462e-9360-0a48161d7245 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:21.986995 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a77f0-8385-462e-9360-0a48161d7245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.007058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-848a77f0-8385-462e-9360-0a48161d7245 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:22.008470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-848a77f0-8385-462e-9360-0a48161d7245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:22.030072 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e5881-8d6a-40a0-8cf8-d11a7b888cb2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:22.067321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:22.077797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f6431-f1d0-4dbe-83d0-4fe3eb170a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:22.079192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f6431-f1d0-4dbe-83d0-4fe3eb170a5e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.079763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f6431-f1d0-4dbe-83d0-4fe3eb170a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.080047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f6431-f1d0-4dbe-83d0-4fe3eb170a5e None admin] PATH_INFO: `/v3/users/f3502a9200a44a11a90434a400f707c9` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.099797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f6431-f1d0-4dbe-83d0-4fe3eb170a5e None admin] RBAC: Authorizing `identity:get_user(user_id=f3502a9200a44a11a90434a400f707c9)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:22.101997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f6431-f1d0-4dbe-83d0-4fe3eb170a5e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:22.105131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 109/218] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:22.108965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 110/219] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:21 2022] GET /identity/v3/users/f3502a9200a44a11a90434a400f707c9 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:22.110342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a77f0-8385-462e-9360-0a48161d7245 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:22.128175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f6431-f1d0-4dbe-83d0-4fe3eb170a5e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:22.230693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a77f0-8385-462e-9360-0a48161d7245 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:22.253971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f6431-f1d0-4dbe-83d0-4fe3eb170a5e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:22.274916 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:22.286117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b410591-14c1-4295-897f-8068d3bfe518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:22.287068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b410591-14c1-4295-897f-8068d3bfe518 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.287356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b410591-14c1-4295-897f-8068d3bfe518 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.287640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b410591-14c1-4295-897f-8068d3bfe518 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.297321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f553d5-8d5b-4e8b-9662-492e88e82d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:22.298257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f553d5-8d5b-4e8b-9662-492e88e82d82 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.298551 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f553d5-8d5b-4e8b-9662-492e88e82d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.298830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f553d5-8d5b-4e8b-9662-492e88e82d82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.313566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f553d5-8d5b-4e8b-9662-492e88e82d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-16f553d5-8d5b-4e8b-9662-492e88e82d82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:22.322851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:22.323772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:22.324538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 111/220] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:49:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:22.327350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf8bac-b6d7-4d7d-bc1d-49c4e28c8452 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.327682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf8bac-b6d7-4d7d-bc1d-49c4e28c8452 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.328128 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf8bac-b6d7-4d7d-bc1d-49c4e28c8452 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.331290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 112/221] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:22.331290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf8bac-b6d7-4d7d-bc1d-49c4e28c8452 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:22.381614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b410591-14c1-4295-897f-8068d3bfe518 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:22.383518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b410591-14c1-4295-897f-8068d3bfe518 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:22.392288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:22.459123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf8bac-b6d7-4d7d-bc1d-49c4e28c8452 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:22.502212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab6326-ed5c-4b3f-a9f7-3dfedc1f9544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:22.502897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab6326-ed5c-4b3f-a9f7-3dfedc1f9544 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.503191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab6326-ed5c-4b3f-a9f7-3dfedc1f9544 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.503473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab6326-ed5c-4b3f-a9f7-3dfedc1f9544 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.504728 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab6326-ed5c-4b3f-a9f7-3dfedc1f9544 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:22.505048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab6326-ed5c-4b3f-a9f7-3dfedc1f9544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:22.506946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab6326-ed5c-4b3f-a9f7-3dfedc1f9544 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:22.518830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 113/222] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:22.520141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8def6-70f6-4806-a321-522982a37680 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.520405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8def6-70f6-4806-a321-522982a37680 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.520755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8def6-70f6-4806-a321-522982a37680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.520970 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 110/223] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:22.524087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b410591-14c1-4295-897f-8068d3bfe518 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:22.546211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e9a8def6-70f6-4806-a321-522982a37680 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:22.547817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e9a8def6-70f6-4806-a321-522982a37680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:22.565447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:22.622931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 114/224] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:22.624331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8def6-70f6-4806-a321-522982a37680 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:22.642318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b410591-14c1-4295-897f-8068d3bfe518 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:22.669808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:22.678576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba5271-6d54-4aa2-8970-fb82129f34e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:22.679456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5271-6d54-4aa2-8970-fb82129f34e9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.679736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5271-6d54-4aa2-8970-fb82129f34e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.680002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5271-6d54-4aa2-8970-fb82129f34e9 None admin] PATH_INFO: `/v3/users/0b02aef84a6345038b983f7e2c63cdee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.681275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba5271-6d54-4aa2-8970-fb82129f34e9 None admin] RBAC: Authorizing `identity:get_user(user_id=0b02aef84a6345038b983f7e2c63cdee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:22.682744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba5271-6d54-4aa2-8970-fb82129f34e9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:22.684754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 111/225] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:22 2022] GET /identity/v3/users/0b02aef84a6345038b983f7e2c63cdee => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:22.685638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba5271-6d54-4aa2-8970-fb82129f34e9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:22.741003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8def6-70f6-4806-a321-522982a37680 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:22.781339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e113d3-67e1-429a-b34b-4acc6c997e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:22.782308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e113d3-67e1-429a-b34b-4acc6c997e8b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.782597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e113d3-67e1-429a-b34b-4acc6c997e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.782874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e113d3-67e1-429a-b34b-4acc6c997e8b None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.784109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e113d3-67e1-429a-b34b-4acc6c997e8b None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:22.785760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e113d3-67e1-429a-b34b-4acc6c997e8b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:22.787885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 115/226] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:22 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:22.788708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e113d3-67e1-429a-b34b-4acc6c997e8b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:22.799489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:22.803785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba5271-6d54-4aa2-8970-fb82129f34e9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:22.843149 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354910c-a55f-4f28-aace-12024d1cf361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:22.843908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354910c-a55f-4f28-aace-12024d1cf361 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.844193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354910c-a55f-4f28-aace-12024d1cf361 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.844470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354910c-a55f-4f28-aace-12024d1cf361 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.856336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354910c-a55f-4f28-aace-12024d1cf361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-f354910c-a55f-4f28-aace-12024d1cf361 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:22.865140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:22.866205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:22.867093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 112/227] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:22.867775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354910c-a55f-4f28-aace-12024d1cf361 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:22.908633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e113d3-67e1-429a-b34b-4acc6c997e8b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:22.919020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:22.946243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f4c34-be08-483b-83f9-d7ec5daaba1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:22.947346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f4c34-be08-483b-83f9-d7ec5daaba1c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:22.947777 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f4c34-be08-483b-83f9-d7ec5daaba1c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:22.948382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f4c34-be08-483b-83f9-d7ec5daaba1c None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:22.949772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f4c34-be08-483b-83f9-d7ec5daaba1c None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:22.952068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f4c34-be08-483b-83f9-d7ec5daaba1c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:22.954469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 116/228] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:22 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:22.955794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f4c34-be08-483b-83f9-d7ec5daaba1c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:22.978870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354910c-a55f-4f28-aace-12024d1cf361 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:23.014449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f24322-f7e3-4dce-ae66-80d1e8dddaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:23.015107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f24322-f7e3-4dce-ae66-80d1e8dddaef None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:23.015389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f24322-f7e3-4dce-ae66-80d1e8dddaef None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:23.015759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f24322-f7e3-4dce-ae66-80d1e8dddaef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:23.016558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f24322-f7e3-4dce-ae66-80d1e8dddaef None admin] RBAC: Adding query filter params (role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=45204248d1df42a29cab699e5a889efe, user.id=f3502a9200a44a11a90434a400f707c9) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:23.016873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f24322-f7e3-4dce-ae66-80d1e8dddaef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:23.018604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f24322-f7e3-4dce-ae66-80d1e8dddaef None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:23.024260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 113/229] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:22 2022] GET /identity/v3/role_assignments?user.id=f3502a9200a44a11a90434a400f707c9&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=301682fc5c92440f874912eabfed6f70 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:23.025038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f24322-f7e3-4dce-ae66-80d1e8dddaef None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:23.066654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f4c34-be08-483b-83f9-d7ec5daaba1c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:23.079305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:23.106466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25bbfc-2450-480f-b3a7-5092a98f52ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:23.107413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25bbfc-2450-480f-b3a7-5092a98f52ae None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:23.107846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25bbfc-2450-480f-b3a7-5092a98f52ae None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:23.108373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25bbfc-2450-480f-b3a7-5092a98f52ae None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:23.109344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25bbfc-2450-480f-b3a7-5092a98f52ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:23.109799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25bbfc-2450-480f-b3a7-5092a98f52ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:23.111482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25bbfc-2450-480f-b3a7-5092a98f52ae None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:23.120781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 117/230] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:23.126584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25bbfc-2450-480f-b3a7-5092a98f52ae None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:23.204442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:23.271650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f24322-f7e3-4dce-ae66-80d1e8dddaef None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:23.276571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25bbfc-2450-480f-b3a7-5092a98f52ae None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:23.330867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942b282-49a4-4b5c-9bea-f4812d7f2a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:23.332929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942b282-49a4-4b5c-9bea-f4812d7f2a01 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:23.333709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942b282-49a4-4b5c-9bea-f4812d7f2a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:23.334615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942b282-49a4-4b5c-9bea-f4812d7f2a01 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe/users/0b02aef84a6345038b983f7e2c63cdee/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:23.335351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bbd13-d2d6-4c62-834e-9dcde866d220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:23.336390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bbd13-d2d6-4c62-834e-9dcde866d220 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:23.337641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bbd13-d2d6-4c62-834e-9dcde866d220 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:23.338529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bbd13-d2d6-4c62-834e-9dcde866d220 None admin] PATH_INFO: `/v3/users/0b02aef84a6345038b983f7e2c63cdee` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:23.339364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942b282-49a4-4b5c-9bea-f4812d7f2a01 None admin] RBAC: Authorizing `identity:create_grant(project_id=45204248d1df42a29cab699e5a889efe, user_id=0b02aef84a6345038b983f7e2c63cdee, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:23.342442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942b282-49a4-4b5c-9bea-f4812d7f2a01 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:23.343396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bbd13-d2d6-4c62-834e-9dcde866d220 None admin] RBAC: Authorizing `identity:get_user(user_id=0b02aef84a6345038b983f7e2c63cdee)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:23.345104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bbd13-d2d6-4c62-834e-9dcde866d220 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:23.349614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 118/231] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:23 2022] GET /identity/v3/users/0b02aef84a6345038b983f7e2c63cdee => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:23.355373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bbd13-d2d6-4c62-834e-9dcde866d220 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:23.372122 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 114/232] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:49:23 2022] PUT /identity/v3/projects/45204248d1df42a29cab699e5a889efe/users/0b02aef84a6345038b983f7e2c63cdee/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:23.497876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bbd13-d2d6-4c62-834e-9dcde866d220 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:23.506117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:23.538158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b859986-77ae-47fa-a6c2-8d39e0ca1793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:23.539279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b859986-77ae-47fa-a6c2-8d39e0ca1793 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:23.539705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b859986-77ae-47fa-a6c2-8d39e0ca1793 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:23.540244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b859986-77ae-47fa-a6c2-8d39e0ca1793 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:23.541791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b859986-77ae-47fa-a6c2-8d39e0ca1793 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:23.543426 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b859986-77ae-47fa-a6c2-8d39e0ca1793 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:23.546023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 119/233] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:23 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:23.551323 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942b282-49a4-4b5c-9bea-f4812d7f2a01 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:23.641699 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:23.662281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942b282-49a4-4b5c-9bea-f4812d7f2a01 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:23.697215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53542f-246b-4e3a-bf31-4f3a2720e8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:23.698173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53542f-246b-4e3a-bf31-4f3a2720e8a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:23.698458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53542f-246b-4e3a-bf31-4f3a2720e8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:23.698735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53542f-246b-4e3a-bf31-4f3a2720e8a4 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe/users/0b02aef84a6345038b983f7e2c63cdee/roles/8184510545604582b267e4d1871ae9bf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:23.701529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53542f-246b-4e3a-bf31-4f3a2720e8a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=45204248d1df42a29cab699e5a889efe, user_id=0b02aef84a6345038b983f7e2c63cdee, role_id=8184510545604582b267e4d1871ae9bf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:23.702992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53542f-246b-4e3a-bf31-4f3a2720e8a4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:23.720372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 115/234] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:49:23 2022] PUT /identity/v3/projects/45204248d1df42a29cab699e5a889efe/users/0b02aef84a6345038b983f7e2c63cdee/roles/8184510545604582b267e4d1871ae9bf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:24.399134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc1847-ba75-4894-864b-940fa812dbcd None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:24.399475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc1847-ba75-4894-864b-940fa812dbcd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:24.399707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc1847-ba75-4894-864b-940fa812dbcd None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:24.401116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 120/235] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:24.406571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730099-28b8-48f2-aeec-9f183b1bb07a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:24.406865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730099-28b8-48f2-aeec-9f183b1bb07a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:24.407143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730099-28b8-48f2-aeec-9f183b1bb07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:24.430527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1e730099-28b8-48f2-aeec-9f183b1bb07a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:24.431740 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1e730099-28b8-48f2-aeec-9f183b1bb07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:24.468296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132c16a-7d0d-48c6-a5ab-e9623214cf7e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:24.468901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132c16a-7d0d-48c6-a5ab-e9623214cf7e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:24.469440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132c16a-7d0d-48c6-a5ab-e9623214cf7e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:24.471000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 121/236] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:24.476447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04438b08-131c-4b69-b07c-569997058ccc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:24.476979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04438b08-131c-4b69-b07c-569997058ccc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:24.477618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04438b08-131c-4b69-b07c-569997058ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:24.512887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-04438b08-131c-4b69-b07c-569997058ccc None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:24.514897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-04438b08-131c-4b69-b07c-569997058ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:24.521119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 116/237] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:24.560838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e730099-28b8-48f2-aeec-9f183b1bb07a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:24.603187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 122/238] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:24.628114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04438b08-131c-4b69-b07c-569997058ccc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:24.669517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:24.701938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e730099-28b8-48f2-aeec-9f183b1bb07a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:24.739037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04438b08-131c-4b69-b07c-569997058ccc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:24.741337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb993074-8760-4ed2-9799-aea5db0bd8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:24.742023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:24.743023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb993074-8760-4ed2-9799-aea5db0bd8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:24.743501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb993074-8760-4ed2-9799-aea5db0bd8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:24.743967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb993074-8760-4ed2-9799-aea5db0bd8d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:24.745464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb993074-8760-4ed2-9799-aea5db0bd8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:24.747230 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb993074-8760-4ed2-9799-aea5db0bd8d8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:24.750303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 117/239] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:24.762201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb993074-8760-4ed2-9799-aea5db0bd8d8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:24.780985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a146aa2-f150-473b-be02-dde9983d5249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:24.781427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a146aa2-f150-473b-be02-dde9983d5249 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:24.781946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a146aa2-f150-473b-be02-dde9983d5249 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:24.782300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a146aa2-f150-473b-be02-dde9983d5249 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:24.801114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a146aa2-f150-473b-be02-dde9983d5249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7a146aa2-f150-473b-be02-dde9983d5249 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:24.801799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:24.802829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:24.803792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:24.805579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 123/240] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:24.811015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a146aa2-f150-473b-be02-dde9983d5249 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:24.917217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a146aa2-f150-473b-be02-dde9983d5249 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:24.941301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb993074-8760-4ed2-9799-aea5db0bd8d8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:24.973934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4b731-704c-4656-a9f7-d75099d5b505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:24.974725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4b731-704c-4656-a9f7-d75099d5b505 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:24.975090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4b731-704c-4656-a9f7-d75099d5b505 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:24.975447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4b731-704c-4656-a9f7-d75099d5b505 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:24.976293 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4b731-704c-4656-a9f7-d75099d5b505 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:24.976681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4b731-704c-4656-a9f7-d75099d5b505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:24.978349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4b731-704c-4656-a9f7-d75099d5b505 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:24.981961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7549dd4-71bc-4df1-b523-8e9191aa9a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:24.982832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7549dd4-71bc-4df1-b523-8e9191aa9a4f None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:24.983284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7549dd4-71bc-4df1-b523-8e9191aa9a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:24.983726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7549dd4-71bc-4df1-b523-8e9191aa9a4f None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:24.984738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7549dd4-71bc-4df1-b523-8e9191aa9a4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:24.986373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7549dd4-71bc-4df1-b523-8e9191aa9a4f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:25.001317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 124/241] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:25.006556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:25.010424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4b731-704c-4656-a9f7-d75099d5b505 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:25.030162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 118/242] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:24 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:25.049944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:25.162860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4b731-704c-4656-a9f7-d75099d5b505 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:25.204654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c422c-f1ae-4fd9-80ae-6af0d8902d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:25.205612 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c422c-f1ae-4fd9-80ae-6af0d8902d42 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:25.205903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c422c-f1ae-4fd9-80ae-6af0d8902d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:25.206189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c422c-f1ae-4fd9-80ae-6af0d8902d42 None admin] PATH_INFO: `/v3/users/69793c611e0a4ef4880ca4080a86c912` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:25.222788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c422c-f1ae-4fd9-80ae-6af0d8902d42 None admin] RBAC: Authorizing `identity:get_user(user_id=69793c611e0a4ef4880ca4080a86c912)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:25.224302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c422c-f1ae-4fd9-80ae-6af0d8902d42 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:25.226626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 125/243] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:25 2022] GET /identity/v3/users/69793c611e0a4ef4880ca4080a86c912 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:25.232471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7549dd4-71bc-4df1-b523-8e9191aa9a4f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:25.343634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7549dd4-71bc-4df1-b523-8e9191aa9a4f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:25.378258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a55027-7f87-4265-b18b-56e40606503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:25.378923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a55027-7f87-4265-b18b-56e40606503a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:25.379213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a55027-7f87-4265-b18b-56e40606503a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:25.379500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a55027-7f87-4265-b18b-56e40606503a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:25.445558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:25.452861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a55027-7f87-4265-b18b-56e40606503a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:25.454390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a55027-7f87-4265-b18b-56e40606503a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:25.557998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 119/244] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:25.564064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c422c-f1ae-4fd9-80ae-6af0d8902d42 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:25.657405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:25.674129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c422c-f1ae-4fd9-80ae-6af0d8902d42 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:25.713890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af0f48-f522-4996-8c21-cf37de0f74f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:25.714763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af0f48-f522-4996-8c21-cf37de0f74f5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:25.715044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af0f48-f522-4996-8c21-cf37de0f74f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:25.715344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af0f48-f522-4996-8c21-cf37de0f74f5 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:25.716603 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af0f48-f522-4996-8c21-cf37de0f74f5 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:25.718088 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af0f48-f522-4996-8c21-cf37de0f74f5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:25.719798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 126/245] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:25 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:25.725744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a55027-7f87-4265-b18b-56e40606503a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:25.797411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4eb7c3-179a-4812-ae37-9b73131a2c7e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:25.797757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4eb7c3-179a-4812-ae37-9b73131a2c7e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:25.797975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4eb7c3-179a-4812-ae37-9b73131a2c7e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:25.799717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 127/246] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:25.810034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab52eb9-3a66-40c4-b393-a9c482324a18 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:25.810339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab52eb9-3a66-40c4-b393-a9c482324a18 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:25.810771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab52eb9-3a66-40c4-b393-a9c482324a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:25.836224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9ab52eb9-3a66-40c4-b393-a9c482324a18 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:25.837738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9ab52eb9-3a66-40c4-b393-a9c482324a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:25.846214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a55027-7f87-4265-b18b-56e40606503a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:25.849407 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:25.890240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69580050-60f9-4455-b81e-61892271f0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:25.890908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69580050-60f9-4455-b81e-61892271f0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:25.891198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69580050-60f9-4455-b81e-61892271f0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:25.891477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69580050-60f9-4455-b81e-61892271f0e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:25.892333 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69580050-60f9-4455-b81e-61892271f0e4 None admin] RBAC: Adding query filter params (role.id=f216586d507240c9b31a20c99b9bd21a, scope.project.id=45204248d1df42a29cab699e5a889efe, user.id=69793c611e0a4ef4880ca4080a86c912) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:25.892648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69580050-60f9-4455-b81e-61892271f0e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:25.894180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69580050-60f9-4455-b81e-61892271f0e4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:25.901488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 120/247] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:25 2022] GET /identity/v3/role_assignments?user.id=69793c611e0a4ef4880ca4080a86c912&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=f216586d507240c9b31a20c99b9bd21a => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:25.945497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 128/248] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:25.971452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69580050-60f9-4455-b81e-61892271f0e4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:26.050700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa28890-eaac-404c-9a26-168d0ff8780f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:26.051032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa28890-eaac-404c-9a26-168d0ff8780f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:26.051388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa28890-eaac-404c-9a26-168d0ff8780f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:26.052863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 129/249] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:26.058809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c526-e51a-49ef-a884-57a835c8b0f5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:26.059150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c526-e51a-49ef-a884-57a835c8b0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:26.059422 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c526-e51a-49ef-a884-57a835c8b0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:26.093507 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9190c526-e51a-49ef-a884-57a835c8b0f5 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:26.094674 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9190c526-e51a-49ef-a884-57a835c8b0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:26.106341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69580050-60f9-4455-b81e-61892271f0e4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:26.159665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17b8c5-4fad-4fbd-8b63-0bd67e3b244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:26.160516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17b8c5-4fad-4fbd-8b63-0bd67e3b244e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:26.160908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17b8c5-4fad-4fbd-8b63-0bd67e3b244e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:26.161235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17b8c5-4fad-4fbd-8b63-0bd67e3b244e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:26.175059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17b8c5-4fad-4fbd-8b63-0bd67e3b244e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:26.176002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 130/250] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6d17b8c5-4fad-4fbd-8b63-0bd67e3b244e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:26.183356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:26.184941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:26.185960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 121/251] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:26.191496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190c526-e51a-49ef-a884-57a835c8b0f5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:26.212410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:26.248725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17b8c5-4fad-4fbd-8b63-0bd67e3b244e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:26.288280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:26.317838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190c526-e51a-49ef-a884-57a835c8b0f5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:26.361412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361554d8-3121-458f-8cd6-6d08d25b2c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:26.362450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361554d8-3121-458f-8cd6-6d08d25b2c18 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:26.362742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361554d8-3121-458f-8cd6-6d08d25b2c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:26.363028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361554d8-3121-458f-8cd6-6d08d25b2c18 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:26.363793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361554d8-3121-458f-8cd6-6d08d25b2c18 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:26.364107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361554d8-3121-458f-8cd6-6d08d25b2c18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:26.365928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361554d8-3121-458f-8cd6-6d08d25b2c18 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:26.377416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 131/252] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:26.385140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361554d8-3121-458f-8cd6-6d08d25b2c18 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:26.405063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17b8c5-4fad-4fbd-8b63-0bd67e3b244e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:26.471397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:26.476767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e31509-4862-4626-a2c6-b75903f12b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:26.477914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e31509-4862-4626-a2c6-b75903f12b34 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:26.478578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e31509-4862-4626-a2c6-b75903f12b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:26.479865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e31509-4862-4626-a2c6-b75903f12b34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:26.499123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361554d8-3121-458f-8cd6-6d08d25b2c18 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:26.505021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e31509-4862-4626-a2c6-b75903f12b34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-45e31509-4862-4626-a2c6-b75903f12b34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:26.522470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:26.523469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:26.524272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:26.529335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 122/253] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:49:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:26.531143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38fdd6-215b-4c72-a230-48ef9cafa1dc None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:26.531645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38fdd6-215b-4c72-a230-48ef9cafa1dc None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:26.532152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38fdd6-215b-4c72-a230-48ef9cafa1dc None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:26.534035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 123/254] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:26.536352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38fdd6-215b-4c72-a230-48ef9cafa1dc None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:26.565908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea679b-dd89-4d01-b17b-6b24316943d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:26.566814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea679b-dd89-4d01-b17b-6b24316943d5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:26.567101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea679b-dd89-4d01-b17b-6b24316943d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:26.567379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea679b-dd89-4d01-b17b-6b24316943d5 None admin] PATH_INFO: `/v3/users/f3502a9200a44a11a90434a400f707c9` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:26.572456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea679b-dd89-4d01-b17b-6b24316943d5 None admin] RBAC: Authorizing `identity:get_user(user_id=f3502a9200a44a11a90434a400f707c9)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:26.574054 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea679b-dd89-4d01-b17b-6b24316943d5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:26.576112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 132/255] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:26 2022] GET /identity/v3/users/f3502a9200a44a11a90434a400f707c9 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:26.581472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72331932-8c90-43b3-8d69-e0f15d278479 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:26.582053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72331932-8c90-43b3-8d69-e0f15d278479 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:26.582053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72331932-8c90-43b3-8d69-e0f15d278479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:26.646338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-72331932-8c90-43b3-8d69-e0f15d278479 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:26.647486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-72331932-8c90-43b3-8d69-e0f15d278479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:26.797483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:26.850477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38fdd6-215b-4c72-a230-48ef9cafa1dc None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:26.863220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 133/256] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:26.863988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72331932-8c90-43b3-8d69-e0f15d278479 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:26.920022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e5bdf-8834-4910-9022-43aef0016dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:26.924095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5bdf-8834-4910-9022-43aef0016dfd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:26.926420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5bdf-8834-4910-9022-43aef0016dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:26.926950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5bdf-8834-4910-9022-43aef0016dfd None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:26.928473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e5bdf-8834-4910-9022-43aef0016dfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:26.929546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e5bdf-8834-4910-9022-43aef0016dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:26.930986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e5bdf-8834-4910-9022-43aef0016dfd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:26.963419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 124/257] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:26.964297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e5bdf-8834-4910-9022-43aef0016dfd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:27.013232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:27.089965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72331932-8c90-43b3-8d69-e0f15d278479 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:27.146211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5d5d7-3a1e-490c-8262-ebe9f096e74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:27.147631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5d5d7-3a1e-490c-8262-ebe9f096e74e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:27.150451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5d5d7-3a1e-490c-8262-ebe9f096e74e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:27.150945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5d5d7-3a1e-490c-8262-ebe9f096e74e None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:27.153876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5d5d7-3a1e-490c-8262-ebe9f096e74e None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:27.156565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5d5d7-3a1e-490c-8262-ebe9f096e74e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:27.158758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 134/258] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:26 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:27.161070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5d5d7-3a1e-490c-8262-ebe9f096e74e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:27.239215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e5bdf-8834-4910-9022-43aef0016dfd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:27.271346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5d5d7-3a1e-490c-8262-ebe9f096e74e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:27.310960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf2ef1-0368-4801-93ca-7662d3526a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:27.312034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf2ef1-0368-4801-93ca-7662d3526a6e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:27.312317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf2ef1-0368-4801-93ca-7662d3526a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:27.312845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf2ef1-0368-4801-93ca-7662d3526a6e None admin] PATH_INFO: `/v3/users/0b02aef84a6345038b983f7e2c63cdee` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:27.314214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf2ef1-0368-4801-93ca-7662d3526a6e None admin] RBAC: Authorizing `identity:get_user(user_id=0b02aef84a6345038b983f7e2c63cdee)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:27.315659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf2ef1-0368-4801-93ca-7662d3526a6e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:27.317336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 135/259] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:27 2022] GET /identity/v3/users/0b02aef84a6345038b983f7e2c63cdee => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:27.318079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf2ef1-0368-4801-93ca-7662d3526a6e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:27.334929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:27.337964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3d16c-ee01-4731-9f20-7490c00e8755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:27.338831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3d16c-ee01-4731-9f20-7490c00e8755 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:27.339121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3d16c-ee01-4731-9f20-7490c00e8755 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:27.339403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3d16c-ee01-4731-9f20-7490c00e8755 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:27.353076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3d16c-ee01-4731-9f20-7490c00e8755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a6e3d16c-ee01-4731-9f20-7490c00e8755 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:27.362807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:27.363772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:27.364551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:27.365649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 125/260] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:27.372819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3d16c-ee01-4731-9f20-7490c00e8755 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:27.378074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:27.538138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3d16c-ee01-4731-9f20-7490c00e8755 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:27.551048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf2ef1-0368-4801-93ca-7662d3526a6e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:27.577073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7617eed-e94e-4f24-95f1-3f92d7e3342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:27.577978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7617eed-e94e-4f24-95f1-3f92d7e3342f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:27.578409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7617eed-e94e-4f24-95f1-3f92d7e3342f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:27.579608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7617eed-e94e-4f24-95f1-3f92d7e3342f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:27.615101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8fe7b-8dab-4a86-a726-aec38a80f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:27.616558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8fe7b-8dab-4a86-a726-aec38a80f936 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:27.618458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8fe7b-8dab-4a86-a726-aec38a80f936 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:27.618906 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8fe7b-8dab-4a86-a726-aec38a80f936 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe/users/f3502a9200a44a11a90434a400f707c9/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:27.626055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8fe7b-8dab-4a86-a726-aec38a80f936 None admin] RBAC: Authorizing `identity:create_grant(project_id=45204248d1df42a29cab699e5a889efe, user_id=f3502a9200a44a11a90434a400f707c9, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:27.627684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8fe7b-8dab-4a86-a726-aec38a80f936 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:27.648934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 136/261] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:49:27 2022] PUT /identity/v3/projects/45204248d1df42a29cab699e5a889efe/users/f3502a9200a44a11a90434a400f707c9/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:27.654654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8fe7b-8dab-4a86-a726-aec38a80f936 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:27.771455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7617eed-e94e-4f24-95f1-3f92d7e3342f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:27.772971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7617eed-e94e-4f24-95f1-3f92d7e3342f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:27.777485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:27.789127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:27.879955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8fe7b-8dab-4a86-a726-aec38a80f936 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:27.954712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a52ed-1748-49ba-a1e4-bd049317a20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:27.955403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a52ed-1748-49ba-a1e4-bd049317a20d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:27.955687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a52ed-1748-49ba-a1e4-bd049317a20d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:27.955960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a52ed-1748-49ba-a1e4-bd049317a20d None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:27.956724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a52ed-1748-49ba-a1e4-bd049317a20d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:27.957033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a52ed-1748-49ba-a1e4-bd049317a20d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:27.958502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a52ed-1748-49ba-a1e4-bd049317a20d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:27.960362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 126/262] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:27.967337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7617eed-e94e-4f24-95f1-3f92d7e3342f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:27.977519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 137/263] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:27.987146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a52ed-1748-49ba-a1e4-bd049317a20d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:28.163946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7617eed-e94e-4f24-95f1-3f92d7e3342f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:28.165463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:28.240694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ff410-ec6b-443c-9d6e-43bc69e91122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:28.241592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ff410-ec6b-443c-9d6e-43bc69e91122 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.241880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ff410-ec6b-443c-9d6e-43bc69e91122 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.242156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ff410-ec6b-443c-9d6e-43bc69e91122 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.245831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ff410-ec6b-443c-9d6e-43bc69e91122 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:28.247464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ff410-ec6b-443c-9d6e-43bc69e91122 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:28.249572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 127/264] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:27 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:28.251853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a52ed-1748-49ba-a1e4-bd049317a20d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:28.255833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ff410-ec6b-443c-9d6e-43bc69e91122 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:28.303581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:28.317846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc907dc-477f-40ef-8d0b-8a7e3c0249ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:28.318788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc907dc-477f-40ef-8d0b-8a7e3c0249ed None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.319039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc907dc-477f-40ef-8d0b-8a7e3c0249ed None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.319325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc907dc-477f-40ef-8d0b-8a7e3c0249ed None admin] PATH_INFO: `/v3/users/0b02aef84a6345038b983f7e2c63cdee` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.320713 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc907dc-477f-40ef-8d0b-8a7e3c0249ed None admin] RBAC: Authorizing `identity:get_user(user_id=0b02aef84a6345038b983f7e2c63cdee)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:28.322219 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc907dc-477f-40ef-8d0b-8a7e3c0249ed None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:28.324239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 138/265] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:27 2022] GET /identity/v3/users/0b02aef84a6345038b983f7e2c63cdee => generated 283 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:28.342335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc907dc-477f-40ef-8d0b-8a7e3c0249ed None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:28.475436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc907dc-477f-40ef-8d0b-8a7e3c0249ed None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:28.487930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ff410-ec6b-443c-9d6e-43bc69e91122 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:28.512061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e78113-587c-4b6b-8057-cba54ec96efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:28.513306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e78113-587c-4b6b-8057-cba54ec96efb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.513596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e78113-587c-4b6b-8057-cba54ec96efb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.513876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e78113-587c-4b6b-8057-cba54ec96efb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.556086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37265600-8d5a-480d-a5dc-059d90f1adc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:28.556768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37265600-8d5a-480d-a5dc-059d90f1adc0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.557052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37265600-8d5a-480d-a5dc-059d90f1adc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.557481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37265600-8d5a-480d-a5dc-059d90f1adc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.558351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37265600-8d5a-480d-a5dc-059d90f1adc0 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=45204248d1df42a29cab699e5a889efe, user.id=0b02aef84a6345038b983f7e2c63cdee) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:28.558666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37265600-8d5a-480d-a5dc-059d90f1adc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:28.560903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37265600-8d5a-480d-a5dc-059d90f1adc0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:28.576028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 128/266] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:28 2022] GET /identity/v3/role_assignments?user.id=0b02aef84a6345038b983f7e2c63cdee&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=e089a0d6040040c4bcf143a680488e3d => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:28.587779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:28.592157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e78113-587c-4b6b-8057-cba54ec96efb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:28.594108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e78113-587c-4b6b-8057-cba54ec96efb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:28.625698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56475-ef38-495b-872e-43f10c4a2322 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.625991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56475-ef38-495b-872e-43f10c4a2322 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.626430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56475-ef38-495b-872e-43f10c4a2322 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.628125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 129/267] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:28.634730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435eae5-aefc-4d59-9334-c81ec102b869 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.635274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435eae5-aefc-4d59-9334-c81ec102b869 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.635693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435eae5-aefc-4d59-9334-c81ec102b869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.674815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c435eae5-aefc-4d59-9334-c81ec102b869 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:28.676287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c435eae5-aefc-4d59-9334-c81ec102b869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:28.716514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 139/268] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:28.723391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e78113-587c-4b6b-8057-cba54ec96efb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:28.747428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:28.799075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 130/269] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:28.805118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5c7c0-31f9-4512-ae97-deae356a1472 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.805990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5c7c0-31f9-4512-ae97-deae356a1472 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.806510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5c7c0-31f9-4512-ae97-deae356a1472 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.808169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 131/270] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:28.823523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a097115-69db-48f5-b394-ccea822d695b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.824035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a097115-69db-48f5-b394-ccea822d695b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.824453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a097115-69db-48f5-b394-ccea822d695b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.848866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e78113-587c-4b6b-8057-cba54ec96efb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:28.850730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:28.854862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9a097115-69db-48f5-b394-ccea822d695b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:28.857128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9a097115-69db-48f5-b394-ccea822d695b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:28.890117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf012e-6bc3-400a-bbf7-565296fdcb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:28.892126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf012e-6bc3-400a-bbf7-565296fdcb1c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.892824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf012e-6bc3-400a-bbf7-565296fdcb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.893539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf012e-6bc3-400a-bbf7-565296fdcb1c None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.894924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf012e-6bc3-400a-bbf7-565296fdcb1c None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:28.896546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf012e-6bc3-400a-bbf7-565296fdcb1c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:28.898598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 140/271] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:28 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:28.899806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aeb26-9373-49e2-981c-50fd5c8fad4f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:28.900459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aeb26-9373-49e2-981c-50fd5c8fad4f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:28.900459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aeb26-9373-49e2-981c-50fd5c8fad4f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:28.906118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 141/272] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:28.906939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aeb26-9373-49e2-981c-50fd5c8fad4f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:28.944958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 132/273] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:28.949108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a097115-69db-48f5-b394-ccea822d695b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:29.045650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aeb26-9373-49e2-981c-50fd5c8fad4f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:29.058922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:29.066570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a097115-69db-48f5-b394-ccea822d695b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:29.085608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bdd85-e90d-4aad-bdf6-c7947ed93db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:29.086407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bdd85-e90d-4aad-bdf6-c7947ed93db7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.086911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bdd85-e90d-4aad-bdf6-c7947ed93db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.087030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bdd85-e90d-4aad-bdf6-c7947ed93db7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.098173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bdd85-e90d-4aad-bdf6-c7947ed93db7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:29.104722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15ada2-cf65-48c8-a934-ec4494f7c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:29.105414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15ada2-cf65-48c8-a934-ec4494f7c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.105704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15ada2-cf65-48c8-a934-ec4494f7c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.106018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15ada2-cf65-48c8-a934-ec4494f7c0c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-cb6bdd85-e90d-4aad-bdf6-c7947ed93db7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:29.106191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:29.107061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:29.112914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:29.113575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15ada2-cf65-48c8-a934-ec4494f7c0c1 None admin] RBAC: Adding query filter params (role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=45204248d1df42a29cab699e5a889efe, user.id=0b02aef84a6345038b983f7e2c63cdee) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:29.113575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15ada2-cf65-48c8-a934-ec4494f7c0c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:29.113697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 142/274] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:29.113761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15ada2-cf65-48c8-a934-ec4494f7c0c1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:29.113824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd2e23-435f-450b-b0da-b61012c875a3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.113824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd2e23-435f-450b-b0da-b61012c875a3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.113824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd2e23-435f-450b-b0da-b61012c875a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.125645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 133/275] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:28 2022] GET /identity/v3/role_assignments?user.id=0b02aef84a6345038b983f7e2c63cdee&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=8184510545604582b267e4d1871ae9bf => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:29.125645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15ada2-cf65-48c8-a934-ec4494f7c0c1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:29.140082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d4fd2e23-435f-450b-b0da-b61012c875a3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:29.141417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d4fd2e23-435f-450b-b0da-b61012c875a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:29.166408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:29.231136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 143/276] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:29.238147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd2e23-435f-450b-b0da-b61012c875a3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:29.272826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15ada2-cf65-48c8-a934-ec4494f7c0c1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:29.310855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1f775-a8b4-4333-9548-ffb8a80db941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:29.311966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1f775-a8b4-4333-9548-ffb8a80db941 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.312303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1f775-a8b4-4333-9548-ffb8a80db941 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.312876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1f775-a8b4-4333-9548-ffb8a80db941 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.314235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1f775-a8b4-4333-9548-ffb8a80db941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:29.315741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1f775-a8b4-4333-9548-ffb8a80db941 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:29.317518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 134/277] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:29.319736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1f775-a8b4-4333-9548-ffb8a80db941 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:29.368684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:29.401655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd2e23-435f-450b-b0da-b61012c875a3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:29.441204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042da60-5564-448d-8d20-4eb5e89b2e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:29.441868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042da60-5564-448d-8d20-4eb5e89b2e4d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.442154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042da60-5564-448d-8d20-4eb5e89b2e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.442436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042da60-5564-448d-8d20-4eb5e89b2e4d None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.443194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042da60-5564-448d-8d20-4eb5e89b2e4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:29.443504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042da60-5564-448d-8d20-4eb5e89b2e4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:29.444901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042da60-5564-448d-8d20-4eb5e89b2e4d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:29.452801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1f775-a8b4-4333-9548-ffb8a80db941 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:29.454334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 144/278] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:29.458822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042da60-5564-448d-8d20-4eb5e89b2e4d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:29.476899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:29.494501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7cc3c-533c-47f6-9bb1-564f84dbcf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:29.495309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7cc3c-533c-47f6-9bb1-564f84dbcf9e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.495597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7cc3c-533c-47f6-9bb1-564f84dbcf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.495870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7cc3c-533c-47f6-9bb1-564f84dbcf9e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.508041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7cc3c-533c-47f6-9bb1-564f84dbcf9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5ae7cc3c-533c-47f6-9bb1-564f84dbcf9e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:29.516531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:29.517491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 135/279] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:29.523444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7cc3c-533c-47f6-9bb1-564f84dbcf9e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:29.575737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042da60-5564-448d-8d20-4eb5e89b2e4d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:29.611158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8dd38-cc0c-4a0d-a966-61b10c3a3f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:29.611826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8dd38-cc0c-4a0d-a966-61b10c3a3f49 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.612107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8dd38-cc0c-4a0d-a966-61b10c3a3f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.612383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8dd38-cc0c-4a0d-a966-61b10c3a3f49 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.613108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8dd38-cc0c-4a0d-a966-61b10c3a3f49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:29.614027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:29.614540 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8dd38-cc0c-4a0d-a966-61b10c3a3f49 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7cc3c-533c-47f6-9bb1-564f84dbcf9e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:29.637804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 145/280] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:29 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:29.646169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8dd38-cc0c-4a0d-a966-61b10c3a3f49 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:29.676127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f71b-6019-4b28-b0d6-4e592ebf3364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:29.677026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f71b-6019-4b28-b0d6-4e592ebf3364 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.677345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f71b-6019-4b28-b0d6-4e592ebf3364 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.677622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f71b-6019-4b28-b0d6-4e592ebf3364 None admin] PATH_INFO: `/v3/users/94edf98909ad47aca0fd0b0702e11163` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.697826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20f71b-6019-4b28-b0d6-4e592ebf3364 None admin] RBAC: Authorizing `identity:get_user(user_id=94edf98909ad47aca0fd0b0702e11163)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:29.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20f71b-6019-4b28-b0d6-4e592ebf3364 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:29.701380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 136/281] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:29 2022] GET /identity/v3/users/94edf98909ad47aca0fd0b0702e11163 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:29.708321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f71b-6019-4b28-b0d6-4e592ebf3364 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:29.771806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:29.804585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8dd38-cc0c-4a0d-a966-61b10c3a3f49 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:29.841619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f71b-6019-4b28-b0d6-4e592ebf3364 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:29.851505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870e7b8-fb97-414e-8ba2-948fb173614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:29.852194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e7b8-fb97-414e-8ba2-948fb173614c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.852483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e7b8-fb97-414e-8ba2-948fb173614c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.852764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e7b8-fb97-414e-8ba2-948fb173614c None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.853924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870e7b8-fb97-414e-8ba2-948fb173614c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:29.856386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870e7b8-fb97-414e-8ba2-948fb173614c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:29.856828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870e7b8-fb97-414e-8ba2-948fb173614c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:29.868782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 146/282] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:29.875546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870e7b8-fb97-414e-8ba2-948fb173614c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:29.884673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f8923-89b5-4dc0-b625-5246b88e7b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:29.885593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f8923-89b5-4dc0-b625-5246b88e7b5e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:29.886008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f8923-89b5-4dc0-b625-5246b88e7b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:29.886425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f8923-89b5-4dc0-b625-5246b88e7b5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:29.887437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f8923-89b5-4dc0-b625-5246b88e7b5e None admin] RBAC: Adding query filter params (role.id=301682fc5c92440f874912eabfed6f70, user.id=94edf98909ad47aca0fd0b0702e11163, scope.system=all) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:29.887920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f8923-89b5-4dc0-b625-5246b88e7b5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:29.889543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f8923-89b5-4dc0-b625-5246b88e7b5e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:29.901307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 137/283] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Jun 21 18:49:29 2022] GET /identity/v3/role_assignments?user.id=94edf98909ad47aca0fd0b0702e11163&scope.system=all&role.id=301682fc5c92440f874912eabfed6f70 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:30.014022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870e7b8-fb97-414e-8ba2-948fb173614c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:30.056929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c25039-5741-4862-bcfc-fdf5ab3ce1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:30.057910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c25039-5741-4862-bcfc-fdf5ab3ce1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:30.058127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c25039-5741-4862-bcfc-fdf5ab3ce1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:30.058407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c25039-5741-4862-bcfc-fdf5ab3ce1e0 None admin] PATH_INFO: `/v3/users/69793c611e0a4ef4880ca4080a86c912` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:30.059676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c25039-5741-4862-bcfc-fdf5ab3ce1e0 None admin] RBAC: Authorizing `identity:get_user(user_id=69793c611e0a4ef4880ca4080a86c912)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:30.061136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c25039-5741-4862-bcfc-fdf5ab3ce1e0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:30.063489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 147/284] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:29 2022] GET /identity/v3/users/69793c611e0a4ef4880ca4080a86c912 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:30.068645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f8923-89b5-4dc0-b625-5246b88e7b5e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:30.080563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:30.151272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638024f-1cb4-408e-a1c7-55793a400e26 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:30.151481 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638024f-1cb4-408e-a1c7-55793a400e26 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:30.151781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638024f-1cb4-408e-a1c7-55793a400e26 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:30.154064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 148/285] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:30.160689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d3183-f713-4b95-9c52-4acfa8f8e3da None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:30.160980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d3183-f713-4b95-9c52-4acfa8f8e3da None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:30.161286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d3183-f713-4b95-9c52-4acfa8f8e3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:30.167392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:30.181441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-194d3183-f713-4b95-9c52-4acfa8f8e3da None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:30.182779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-194d3183-f713-4b95-9c52-4acfa8f8e3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:30.190565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f8923-89b5-4dc0-b625-5246b88e7b5e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:30.236794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bbb4a-7d8a-442a-ae3c-05051528784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:30.237722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bbb4a-7d8a-442a-ae3c-05051528784b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:30.238012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bbb4a-7d8a-442a-ae3c-05051528784b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:30.238297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bbb4a-7d8a-442a-ae3c-05051528784b None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:30.245846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bbb4a-7d8a-442a-ae3c-05051528784b None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:30.247877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bbb4a-7d8a-442a-ae3c-05051528784b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:30.249991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 138/286] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:30 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:30.256145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bbb4a-7d8a-442a-ae3c-05051528784b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:30.295075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 149/287] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:30.355198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d3183-f713-4b95-9c52-4acfa8f8e3da None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:30.397588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:30.439212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bbb4a-7d8a-442a-ae3c-05051528784b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:30.505305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc42c22-e72b-40ee-a3ef-74883801f116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:30.506228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc42c22-e72b-40ee-a3ef-74883801f116 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:30.506516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc42c22-e72b-40ee-a3ef-74883801f116 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:30.506795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc42c22-e72b-40ee-a3ef-74883801f116 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe/users/69793c611e0a4ef4880ca4080a86c912/roles/f216586d507240c9b31a20c99b9bd21a` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:30.510079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc42c22-e72b-40ee-a3ef-74883801f116 None admin] RBAC: Authorizing `identity:create_grant(project_id=45204248d1df42a29cab699e5a889efe, user_id=69793c611e0a4ef4880ca4080a86c912, role_id=f216586d507240c9b31a20c99b9bd21a)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:30.512425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc42c22-e72b-40ee-a3ef-74883801f116 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:30.520885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d3183-f713-4b95-9c52-4acfa8f8e3da None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:30.540640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 139/288] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:49:30 2022] PUT /identity/v3/projects/45204248d1df42a29cab699e5a889efe/users/69793c611e0a4ef4880ca4080a86c912/roles/f216586d507240c9b31a20c99b9bd21a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:49:30.559756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831ac99-e06a-4e62-a5d7-680b222bf5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:30.560641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ac99-e06a-4e62-a5d7-680b222bf5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:30.560985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ac99-e06a-4e62-a5d7-680b222bf5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:30.561360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ac99-e06a-4e62-a5d7-680b222bf5b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:30.563932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:30.574941 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831ac99-e06a-4e62-a5d7-680b222bf5b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-c831ac99-e06a-4e62-a5d7-680b222bf5b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:30.583521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:30.584445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:30.585548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 150/289] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:30.591021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc42c22-e72b-40ee-a3ef-74883801f116 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:30.727356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc42c22-e72b-40ee-a3ef-74883801f116 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:30.766208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff870a-72c4-4d00-a137-dc7360db9038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:30.767259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff870a-72c4-4d00-a137-dc7360db9038 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:30.767744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff870a-72c4-4d00-a137-dc7360db9038 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:30.768218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff870a-72c4-4d00-a137-dc7360db9038 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:30.769216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff870a-72c4-4d00-a137-dc7360db9038 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:30.769752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff870a-72c4-4d00-a137-dc7360db9038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:30.771355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff870a-72c4-4d00-a137-dc7360db9038 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:30.774281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:30.782163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 140/290] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:30.788186 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831ac99-e06a-4e62-a5d7-680b222bf5b8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:30.907472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831ac99-e06a-4e62-a5d7-680b222bf5b8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:30.957852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ed4b0-941e-4149-96de-9cbf67272430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:30.958730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ed4b0-941e-4149-96de-9cbf67272430 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:30.959012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ed4b0-941e-4149-96de-9cbf67272430 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:30.959285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ed4b0-941e-4149-96de-9cbf67272430 None admin] PATH_INFO: `/v3/users/f3502a9200a44a11a90434a400f707c9` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:30.960479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ed4b0-941e-4149-96de-9cbf67272430 None admin] RBAC: Authorizing `identity:get_user(user_id=f3502a9200a44a11a90434a400f707c9)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:30.961934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ed4b0-941e-4149-96de-9cbf67272430 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:30.963864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 151/291] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:30 2022] GET /identity/v3/users/f3502a9200a44a11a90434a400f707c9 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:30.969854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff870a-72c4-4d00-a137-dc7360db9038 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:31.102777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff870a-72c4-4d00-a137-dc7360db9038 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:31.138877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561bea8-4f1d-4225-800b-cfa89f09b559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:31.139535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561bea8-4f1d-4225-800b-cfa89f09b559 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:31.139822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561bea8-4f1d-4225-800b-cfa89f09b559 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:31.140099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561bea8-4f1d-4225-800b-cfa89f09b559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:31.208850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:31.213338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561bea8-4f1d-4225-800b-cfa89f09b559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:31.216914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561bea8-4f1d-4225-800b-cfa89f09b559 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:31.339264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 141/292] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:31.352640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ed4b0-941e-4149-96de-9cbf67272430 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:31.450738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:31.506834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ed4b0-941e-4149-96de-9cbf67272430 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:31.554754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9752e8-310e-4642-b430-7c4202cabac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:31.555679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9752e8-310e-4642-b430-7c4202cabac5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:31.555961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9752e8-310e-4642-b430-7c4202cabac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:31.556288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9752e8-310e-4642-b430-7c4202cabac5 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:31.557501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9752e8-310e-4642-b430-7c4202cabac5 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:31.559479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9752e8-310e-4642-b430-7c4202cabac5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:31.565433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 152/293] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:31 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:31.571393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561bea8-4f1d-4225-800b-cfa89f09b559 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:31.686401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:31.690612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561bea8-4f1d-4225-800b-cfa89f09b559 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:31.744689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d12f3-7d2c-4f3c-8cc3-63c2121c47be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:31.745299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d12f3-7d2c-4f3c-8cc3-63c2121c47be None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:31.745588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d12f3-7d2c-4f3c-8cc3-63c2121c47be None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:31.745864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d12f3-7d2c-4f3c-8cc3-63c2121c47be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:31.747250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d12f3-7d2c-4f3c-8cc3-63c2121c47be None admin] RBAC: Adding query filter params (role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=45204248d1df42a29cab699e5a889efe, user.id=f3502a9200a44a11a90434a400f707c9) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:31.747567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d12f3-7d2c-4f3c-8cc3-63c2121c47be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:31.749037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d12f3-7d2c-4f3c-8cc3-63c2121c47be None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:31.762766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 142/294] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:31 2022] GET /identity/v3/role_assignments?user.id=f3502a9200a44a11a90434a400f707c9&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=301682fc5c92440f874912eabfed6f70 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:32.374558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a677a2-9682-4c86-83eb-042740d565f5 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.375538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a677a2-9682-4c86-83eb-042740d565f5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.376022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a677a2-9682-4c86-83eb-042740d565f5 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.377909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 153/295] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:32.383376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c246b75-c52e-411c-9f0b-78a67413b218 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.384211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c246b75-c52e-411c-9f0b-78a67413b218 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.384714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c246b75-c52e-411c-9f0b-78a67413b218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.391218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b656-f16f-4315-a780-44c0a2269074 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.391507 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b656-f16f-4315-a780-44c0a2269074 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.391788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b656-f16f-4315-a780-44c0a2269074 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.393199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 154/296] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:32.398946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704548e-f6d8-4dc5-9c6b-18560d18433d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.399236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704548e-f6d8-4dc5-9c6b-18560d18433d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.399543 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704548e-f6d8-4dc5-9c6b-18560d18433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.410228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6c246b75-c52e-411c-9f0b-78a67413b218 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:32.411746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6c246b75-c52e-411c-9f0b-78a67413b218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:32.424071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7704548e-f6d8-4dc5-9c6b-18560d18433d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:32.425700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7704548e-f6d8-4dc5-9c6b-18560d18433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:32.494131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 143/297] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:32.497781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c1e2c-5b77-4264-86b7-8cc564478183 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.497781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c1e2c-5b77-4264-86b7-8cc564478183 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.497781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c1e2c-5b77-4264-86b7-8cc564478183 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.497781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 144/298] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:32.506835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85dbf1-4ccd-45cc-b241-3a3b5463831f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.507121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85dbf1-4ccd-45cc-b241-3a3b5463831f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.507397 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85dbf1-4ccd-45cc-b241-3a3b5463831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.526109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5f85dbf1-4ccd-45cc-b241-3a3b5463831f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:32.528584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5f85dbf1-4ccd-45cc-b241-3a3b5463831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:32.532058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 155/299] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:32.532693 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704548e-f6d8-4dc5-9c6b-18560d18433d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:32.598867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:32.608845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 145/300] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:32.609748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85dbf1-4ccd-45cc-b241-3a3b5463831f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:32.642373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:32.650342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704548e-f6d8-4dc5-9c6b-18560d18433d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:32.686669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3309801-0994-41c9-850f-221b88eead7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:32.687733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3309801-0994-41c9-850f-221b88eead7e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.688020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3309801-0994-41c9-850f-221b88eead7e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.688415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3309801-0994-41c9-850f-221b88eead7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.689621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3309801-0994-41c9-850f-221b88eead7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:32.691081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3309801-0994-41c9-850f-221b88eead7e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:32.692875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 156/301] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:32.697943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3309801-0994-41c9-850f-221b88eead7e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:32.727349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85dbf1-4ccd-45cc-b241-3a3b5463831f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:32.771007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1939a5c-816b-4319-b28a-90b3fac0b443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:32.771806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1939a5c-816b-4319-b28a-90b3fac0b443 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.772092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1939a5c-816b-4319-b28a-90b3fac0b443 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.772373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1939a5c-816b-4319-b28a-90b3fac0b443 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.783820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1939a5c-816b-4319-b28a-90b3fac0b443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a1939a5c-816b-4319-b28a-90b3fac0b443 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:32.791565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:32.792601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:32.793480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 146/302] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:32.794132 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1939a5c-816b-4319-b28a-90b3fac0b443 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:32.814035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3309801-0994-41c9-850f-221b88eead7e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:32.844738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:32.854120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f1fce-3d81-4009-b48c-2f65a765f34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:32.854856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1fce-3d81-4009-b48c-2f65a765f34a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.855138 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1fce-3d81-4009-b48c-2f65a765f34a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.855413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1fce-3d81-4009-b48c-2f65a765f34a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.866548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f1fce-3d81-4009-b48c-2f65a765f34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5a2f1fce-3d81-4009-b48c-2f65a765f34a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:32.874787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:32.875714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:32.876529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:32.881031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 157/303] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:32.881724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f1fce-3d81-4009-b48c-2f65a765f34a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:32.888231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:32.910180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1939a5c-816b-4319-b28a-90b3fac0b443 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:32.950891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465adf90-cea8-43d8-99bf-ee294c38f09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:32.952549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465adf90-cea8-43d8-99bf-ee294c38f09b None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:32.952549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465adf90-cea8-43d8-99bf-ee294c38f09b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:32.952549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465adf90-cea8-43d8-99bf-ee294c38f09b None admin] PATH_INFO: `/v3/groups` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:32.952817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465adf90-cea8-43d8-99bf-ee294c38f09b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:32.954337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465adf90-cea8-43d8-99bf-ee294c38f09b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:32.973131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 147/304] 10.222.0.70 () {60 vars in 1134 bytes} [Tue Jun 21 18:49:32 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:32.973131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465adf90-cea8-43d8-99bf-ee294c38f09b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:33.021146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f1fce-3d81-4009-b48c-2f65a765f34a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:33.062970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85683e-a7da-4257-b113-a632469b618b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:33.064035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85683e-a7da-4257-b113-a632469b618b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.064522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85683e-a7da-4257-b113-a632469b618b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.065009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85683e-a7da-4257-b113-a632469b618b None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.066019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85683e-a7da-4257-b113-a632469b618b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:33.066532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85683e-a7da-4257-b113-a632469b618b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:33.068240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85683e-a7da-4257-b113-a632469b618b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:33.084115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 158/305] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:33.084115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22052385-885a-4087-a564-1f864db0942d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.084115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22052385-885a-4087-a564-1f864db0942d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.084115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22052385-885a-4087-a564-1f864db0942d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.085669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 159/306] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:33.090111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22052385-885a-4087-a564-1f864db0942d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:33.102612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465adf90-cea8-43d8-99bf-ee294c38f09b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:33.140586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571d5ef-7a1b-4b8d-899c-d9270da60d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:33.141217 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571d5ef-7a1b-4b8d-899c-d9270da60d4f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.141509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571d5ef-7a1b-4b8d-899c-d9270da60d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.141783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571d5ef-7a1b-4b8d-899c-d9270da60d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.142541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571d5ef-7a1b-4b8d-899c-d9270da60d4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:33.142850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571d5ef-7a1b-4b8d-899c-d9270da60d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:33.144227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571d5ef-7a1b-4b8d-899c-d9270da60d4f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:33.150866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:33.159913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 148/307] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:33.161332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187405d-bf59-4b06-b176-b18cf7d8fc60 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.161627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187405d-bf59-4b06-b176-b18cf7d8fc60 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.161904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187405d-bf59-4b06-b176-b18cf7d8fc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.167787 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:33.181743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8187405d-bf59-4b06-b176-b18cf7d8fc60 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:33.183089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8187405d-bf59-4b06-b176-b18cf7d8fc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:33.219097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22052385-885a-4087-a564-1f864db0942d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:33.256394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950c5bf-fd64-417f-bd14-aac5533ab020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:33.257303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950c5bf-fd64-417f-bd14-aac5533ab020 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.257593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950c5bf-fd64-417f-bd14-aac5533ab020 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.257877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950c5bf-fd64-417f-bd14-aac5533ab020 None admin] PATH_INFO: `/v3/users/c6528ca55c914dee98277eb6760f1cdd` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.265200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 149/308] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:33.265200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187405d-bf59-4b06-b176-b18cf7d8fc60 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:33.284928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950c5bf-fd64-417f-bd14-aac5533ab020 None admin] RBAC: Authorizing `identity:get_user(user_id=c6528ca55c914dee98277eb6760f1cdd)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:33.286455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950c5bf-fd64-417f-bd14-aac5533ab020 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:33.288205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 160/309] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:33 2022] GET /identity/v3/users/c6528ca55c914dee98277eb6760f1cdd => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:33.288979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950c5bf-fd64-417f-bd14-aac5533ab020 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:33.380503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187405d-bf59-4b06-b176-b18cf7d8fc60 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:33.407190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950c5bf-fd64-417f-bd14-aac5533ab020 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:33.415455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ce349-2bc5-49b3-80df-797cf08d50f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:33.416423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ce349-2bc5-49b3-80df-797cf08d50f0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.416710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ce349-2bc5-49b3-80df-797cf08d50f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.416999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ce349-2bc5-49b3-80df-797cf08d50f0 None admin] PATH_INFO: `/v3/users/94edf98909ad47aca0fd0b0702e11163` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.418281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ce349-2bc5-49b3-80df-797cf08d50f0 None admin] RBAC: Authorizing `identity:get_user(user_id=94edf98909ad47aca0fd0b0702e11163)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:33.419267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:33.419742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ce349-2bc5-49b3-80df-797cf08d50f0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:33.421650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 150/310] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:33 2022] GET /identity/v3/users/94edf98909ad47aca0fd0b0702e11163 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:33.426768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ce349-2bc5-49b3-80df-797cf08d50f0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:33.429515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:33.448319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c29328-3eea-4e41-85b1-21efc74352cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:33.449139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c29328-3eea-4e41-85b1-21efc74352cb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.449951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c29328-3eea-4e41-85b1-21efc74352cb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.450235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c29328-3eea-4e41-85b1-21efc74352cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.461289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c29328-3eea-4e41-85b1-21efc74352cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b2c29328-3eea-4e41-85b1-21efc74352cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:33.469519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:33.470428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:33.471236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 161/311] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:33.471940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c29328-3eea-4e41-85b1-21efc74352cb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:33.539256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ce349-2bc5-49b3-80df-797cf08d50f0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:33.577281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c29328-3eea-4e41-85b1-21efc74352cb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:33.577614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895c42d-e9ac-4777-8cb5-8ecb5535ace5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:33.578490 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895c42d-e9ac-4777-8cb5-8ecb5535ace5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.579029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895c42d-e9ac-4777-8cb5-8ecb5535ace5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.579610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895c42d-e9ac-4777-8cb5-8ecb5535ace5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.580447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895c42d-e9ac-4777-8cb5-8ecb5535ace5 None admin] RBAC: Adding query filter params (role.id=f216586d507240c9b31a20c99b9bd21a, user.id=c6528ca55c914dee98277eb6760f1cdd, scope.system=all) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:33.580760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895c42d-e9ac-4777-8cb5-8ecb5535ace5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:33.582224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895c42d-e9ac-4777-8cb5-8ecb5535ace5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:33.587853 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 151/312] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Jun 21 18:49:33 2022] GET /identity/v3/role_assignments?user.id=c6528ca55c914dee98277eb6760f1cdd&scope.system=all&role.id=f216586d507240c9b31a20c99b9bd21a => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:33.588691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895c42d-e9ac-4777-8cb5-8ecb5535ace5 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:33.632496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc6ffb-f714-4c62-b7ac-e93602bad53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:33.633789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc6ffb-f714-4c62-b7ac-e93602bad53c None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.634305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc6ffb-f714-4c62-b7ac-e93602bad53c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.634797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc6ffb-f714-4c62-b7ac-e93602bad53c None admin] PATH_INFO: `/v3/system/users/94edf98909ad47aca0fd0b0702e11163/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.638105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc6ffb-f714-4c62-b7ac-e93602bad53c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=94edf98909ad47aca0fd0b0702e11163, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:33.641000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc6ffb-f714-4c62-b7ac-e93602bad53c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:33.656194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 162/313] 10.222.0.70 () {58 vars in 1334 bytes} [Tue Jun 21 18:49:33 2022] PUT /identity/v3/system/users/94edf98909ad47aca0fd0b0702e11163/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:33.668913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:33.714123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895c42d-e9ac-4777-8cb5-8ecb5535ace5 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:33.755229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951ed52-d203-4a22-b804-212dac9ab90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:33.756211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951ed52-d203-4a22-b804-212dac9ab90e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.756653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951ed52-d203-4a22-b804-212dac9ab90e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.757120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951ed52-d203-4a22-b804-212dac9ab90e None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.758036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951ed52-d203-4a22-b804-212dac9ab90e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:33.758513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951ed52-d203-4a22-b804-212dac9ab90e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:33.760085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951ed52-d203-4a22-b804-212dac9ab90e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:33.771005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 152/314] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:33.777166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc6ffb-f714-4c62-b7ac-e93602bad53c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:33.820117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:33.900657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc6ffb-f714-4c62-b7ac-e93602bad53c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:33.936038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe8d94-be39-4096-9b97-ded56208ba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:33.936919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe8d94-be39-4096-9b97-ded56208ba58 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:33.937227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe8d94-be39-4096-9b97-ded56208ba58 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:33.937754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe8d94-be39-4096-9b97-ded56208ba58 None admin] PATH_INFO: `/v3/users/69793c611e0a4ef4880ca4080a86c912` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:33.938945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe8d94-be39-4096-9b97-ded56208ba58 None admin] RBAC: Authorizing `identity:get_user(user_id=69793c611e0a4ef4880ca4080a86c912)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:33.940383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe8d94-be39-4096-9b97-ded56208ba58 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:33.942434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 163/315] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:33 2022] GET /identity/v3/users/69793c611e0a4ef4880ca4080a86c912 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:33.947567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951ed52-d203-4a22-b804-212dac9ab90e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:34.055380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951ed52-d203-4a22-b804-212dac9ab90e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:34.087907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:34.097562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54569d9c-cac0-4426-87dd-f035af5fd949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:34.098215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54569d9c-cac0-4426-87dd-f035af5fd949 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:34.098499 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54569d9c-cac0-4426-87dd-f035af5fd949 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:34.098779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54569d9c-cac0-4426-87dd-f035af5fd949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:34.177820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54569d9c-cac0-4426-87dd-f035af5fd949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:34.179302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54569d9c-cac0-4426-87dd-f035af5fd949 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:34.285804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 153/316] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:34.291264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe8d94-be39-4096-9b97-ded56208ba58 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:34.401804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:34.405804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe8d94-be39-4096-9b97-ded56208ba58 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:34.442918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1355ee1-2585-499c-ac53-55732671b793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:34.443779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1355ee1-2585-499c-ac53-55732671b793 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:34.444061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1355ee1-2585-499c-ac53-55732671b793 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:34.444336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1355ee1-2585-499c-ac53-55732671b793 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:34.445548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1355ee1-2585-499c-ac53-55732671b793 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:34.446982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1355ee1-2585-499c-ac53-55732671b793 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:34.448980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 164/317] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:34 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:34.455172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54569d9c-cac0-4426-87dd-f035af5fd949 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:34.483800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:34.560941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54569d9c-cac0-4426-87dd-f035af5fd949 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:34.602004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ac00a-e66e-4df2-845c-053d1982c2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:34.602666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac00a-e66e-4df2-845c-053d1982c2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:34.602951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac00a-e66e-4df2-845c-053d1982c2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:34.603228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac00a-e66e-4df2-845c-053d1982c2b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:34.604073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac00a-e66e-4df2-845c-053d1982c2b2 None admin] RBAC: Adding query filter params (role.id=f216586d507240c9b31a20c99b9bd21a, scope.project.id=45204248d1df42a29cab699e5a889efe, user.id=69793c611e0a4ef4880ca4080a86c912) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:34.604389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac00a-e66e-4df2-845c-053d1982c2b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:34.605906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac00a-e66e-4df2-845c-053d1982c2b2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:34.612788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 154/318] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:49:34 2022] GET /identity/v3/role_assignments?user.id=69793c611e0a4ef4880ca4080a86c912&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=f216586d507240c9b31a20c99b9bd21a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:35.129680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a0d70-2f9f-40c0-b0a1-412c0e56cbea None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:35.130063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a0d70-2f9f-40c0-b0a1-412c0e56cbea None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:35.130238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a0d70-2f9f-40c0-b0a1-412c0e56cbea None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:35.131766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 165/319] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:35.136758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260e850-cea4-42de-b750-adf6525159da None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:35.137100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260e850-cea4-42de-b750-adf6525159da None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:35.137385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260e850-cea4-42de-b750-adf6525159da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:35.154626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9260e850-cea4-42de-b750-adf6525159da None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:35.155822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9260e850-cea4-42de-b750-adf6525159da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:35.226514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 155/320] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:35.250301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a0d70-2f9f-40c0-b0a1-412c0e56cbea None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:35.354780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a0d70-2f9f-40c0-b0a1-412c0e56cbea None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:35.390665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dadca-d17c-48aa-a238-e88c50433666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:35.391385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dadca-d17c-48aa-a238-e88c50433666 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:35.391663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dadca-d17c-48aa-a238-e88c50433666 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:35.391942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dadca-d17c-48aa-a238-e88c50433666 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:35.393117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dadca-d17c-48aa-a238-e88c50433666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:35.394556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dadca-d17c-48aa-a238-e88c50433666 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:35.400624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 166/321] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:35.401808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260e850-cea4-42de-b750-adf6525159da None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:35.428875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:35.505990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260e850-cea4-42de-b750-adf6525159da None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:35.543961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c00bb-e7ea-420a-a251-fb4ce9120a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:35.544696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c00bb-e7ea-420a-a251-fb4ce9120a5c None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:35.544984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c00bb-e7ea-420a-a251-fb4ce9120a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:35.545291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c00bb-e7ea-420a-a251-fb4ce9120a5c None admin] PATH_INFO: `/v3/groups` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:35.546052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c00bb-e7ea-420a-a251-fb4ce9120a5c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:35.547547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c00bb-e7ea-420a-a251-fb4ce9120a5c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:35.569133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 156/322] 10.222.0.70 () {60 vars in 1134 bytes} [Tue Jun 21 18:49:35 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:35.686429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45482916-e816-4c12-996b-0a813a8d964f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:35.686988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45482916-e816-4c12-996b-0a813a8d964f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:35.686988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45482916-e816-4c12-996b-0a813a8d964f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:35.688825 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 167/323] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:35.694394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67cfe1-0438-4022-bd31-36a002c7ce33 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:35.694677 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67cfe1-0438-4022-bd31-36a002c7ce33 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:35.694944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67cfe1-0438-4022-bd31-36a002c7ce33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:35.712177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5a67cfe1-0438-4022-bd31-36a002c7ce33 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:35.713403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5a67cfe1-0438-4022-bd31-36a002c7ce33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:35.789660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 157/324] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:35.804447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a14a3b-0d5a-4072-89a0-ab32881d0963 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:35.804767 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a14a3b-0d5a-4072-89a0-ab32881d0963 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:35.805008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a14a3b-0d5a-4072-89a0-ab32881d0963 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:35.806576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 168/325] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:35.812931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3623a-11c5-47cd-aa7e-c9970cb9de4a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:35.813257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3623a-11c5-47cd-aa7e-c9970cb9de4a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:35.814805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3623a-11c5-47cd-aa7e-c9970cb9de4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:35.814925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a14a3b-0d5a-4072-89a0-ab32881d0963 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:35.832362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0bc3623a-11c5-47cd-aa7e-c9970cb9de4a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:35.833640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0bc3623a-11c5-47cd-aa7e-c9970cb9de4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:35.846014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:35.864465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:35.925173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 158/326] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:35.945981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a14a3b-0d5a-4072-89a0-ab32881d0963 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:35.949310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3623a-11c5-47cd-aa7e-c9970cb9de4a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:35.990256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9dbe91-e07c-4784-a1d5-017fb42e062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:35.991060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9dbe91-e07c-4784-a1d5-017fb42e062e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:35.991342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9dbe91-e07c-4784-a1d5-017fb42e062e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:35.991619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9dbe91-e07c-4784-a1d5-017fb42e062e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:36.003394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9dbe91-e07c-4784-a1d5-017fb42e062e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-fd9dbe91-e07c-4784-a1d5-017fb42e062e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:36.010911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:36.011873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:36.012663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 169/327] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:36.019429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9dbe91-e07c-4784-a1d5-017fb42e062e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:36.079446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3623a-11c5-47cd-aa7e-c9970cb9de4a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:36.125793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:36.127283 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d189339-dc40-4932-848c-df2cd3a762b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:36.128013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d189339-dc40-4932-848c-df2cd3a762b9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:36.128300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d189339-dc40-4932-848c-df2cd3a762b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:36.128628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d189339-dc40-4932-848c-df2cd3a762b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:36.136385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9dbe91-e07c-4784-a1d5-017fb42e062e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:36.145305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d189339-dc40-4932-848c-df2cd3a762b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5d189339-dc40-4932-848c-df2cd3a762b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:36.156890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:36.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:36.158649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:36.161959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 159/328] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:36.168992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d189339-dc40-4932-848c-df2cd3a762b9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:36.173677 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:36.176764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b7769-c9c9-47d3-a395-2c28a23985b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:36.177154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b7769-c9c9-47d3-a395-2c28a23985b8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:36.177464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b7769-c9c9-47d3-a395-2c28a23985b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:36.177748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b7769-c9c9-47d3-a395-2c28a23985b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:36.178587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b7769-c9c9-47d3-a395-2c28a23985b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:36.178955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b7769-c9c9-47d3-a395-2c28a23985b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:36.180423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b7769-c9c9-47d3-a395-2c28a23985b8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:36.190629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 170/329] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:36.195759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b7769-c9c9-47d3-a395-2c28a23985b8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:36.310503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d189339-dc40-4932-848c-df2cd3a762b9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:36.328203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b7769-c9c9-47d3-a395-2c28a23985b8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:36.371569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4f01e-c42a-4b45-88a1-efc70ec8abb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:36.372243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4f01e-c42a-4b45-88a1-efc70ec8abb7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:36.372537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4f01e-c42a-4b45-88a1-efc70ec8abb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:36.372823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4f01e-c42a-4b45-88a1-efc70ec8abb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:36.373852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4f01e-c42a-4b45-88a1-efc70ec8abb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:36.374141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cf8fb-8ef7-4531-a3a2-5da0cc36b3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:36.374255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4f01e-c42a-4b45-88a1-efc70ec8abb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:36.375306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cf8fb-8ef7-4531-a3a2-5da0cc36b3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:36.375549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4f01e-c42a-4b45-88a1-efc70ec8abb7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:36.376008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cf8fb-8ef7-4531-a3a2-5da0cc36b3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:36.376486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cf8fb-8ef7-4531-a3a2-5da0cc36b3f7 None admin] PATH_INFO: `/v3/users/c6528ca55c914dee98277eb6760f1cdd` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:36.378147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cf8fb-8ef7-4531-a3a2-5da0cc36b3f7 None admin] RBAC: Authorizing `identity:get_user(user_id=c6528ca55c914dee98277eb6760f1cdd)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:36.379787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cf8fb-8ef7-4531-a3a2-5da0cc36b3f7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:36.382076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 171/330] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:36 2022] GET /identity/v3/users/c6528ca55c914dee98277eb6760f1cdd => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:36.384708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 160/331] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:36.390153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cf8fb-8ef7-4531-a3a2-5da0cc36b3f7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:36.391067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4f01e-c42a-4b45-88a1-efc70ec8abb7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:36.422360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:36.459554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:36.500858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4f01e-c42a-4b45-88a1-efc70ec8abb7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:36.512458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cf8fb-8ef7-4531-a3a2-5da0cc36b3f7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:36.538240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08550fc0-90f3-4e53-b6de-4ab7ba8db900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:36.539126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08550fc0-90f3-4e53-b6de-4ab7ba8db900 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:36.539421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08550fc0-90f3-4e53-b6de-4ab7ba8db900 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:36.539684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08550fc0-90f3-4e53-b6de-4ab7ba8db900 None admin] PATH_INFO: `/v3/users/94edf98909ad47aca0fd0b0702e11163` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:36.540940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08550fc0-90f3-4e53-b6de-4ab7ba8db900 None admin] RBAC: Authorizing `identity:get_user(user_id=94edf98909ad47aca0fd0b0702e11163)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:36.542449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08550fc0-90f3-4e53-b6de-4ab7ba8db900 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:36.544440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 161/332] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:36 2022] GET /identity/v3/users/94edf98909ad47aca0fd0b0702e11163 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:36.550698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08550fc0-90f3-4e53-b6de-4ab7ba8db900 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:36.551270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb0521-675d-4b66-aeea-d78bf53e737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:36.552415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb0521-675d-4b66-aeea-d78bf53e737a None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:36.552892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb0521-675d-4b66-aeea-d78bf53e737a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:36.553504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb0521-675d-4b66-aeea-d78bf53e737a None admin] PATH_INFO: `/v3/system/users/c6528ca55c914dee98277eb6760f1cdd/roles/f216586d507240c9b31a20c99b9bd21a` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:36.556040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb0521-675d-4b66-aeea-d78bf53e737a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c6528ca55c914dee98277eb6760f1cdd, role_id=f216586d507240c9b31a20c99b9bd21a)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:36.557697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb0521-675d-4b66-aeea-d78bf53e737a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:36.568232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 172/333] 10.222.0.70 () {58 vars in 1334 bytes} [Tue Jun 21 18:49:36 2022] PUT /identity/v3/system/users/c6528ca55c914dee98277eb6760f1cdd/roles/f216586d507240c9b31a20c99b9bd21a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:36.668422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08550fc0-90f3-4e53-b6de-4ab7ba8db900 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:36.703380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:36.708464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefacfb-cd3d-4da0-bf7e-51af49e80a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:36.709171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefacfb-cd3d-4da0-bf7e-51af49e80a37 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:36.709464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefacfb-cd3d-4da0-bf7e-51af49e80a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:36.709749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefacfb-cd3d-4da0-bf7e-51af49e80a37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:36.710674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefacfb-cd3d-4da0-bf7e-51af49e80a37 None admin] RBAC: Adding query filter params (role.id=301682fc5c92440f874912eabfed6f70, user.id=94edf98909ad47aca0fd0b0702e11163, scope.system=all) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:36.710929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefacfb-cd3d-4da0-bf7e-51af49e80a37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:36.712748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefacfb-cd3d-4da0-bf7e-51af49e80a37 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:36.726828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 162/334] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Jun 21 18:49:36 2022] GET /identity/v3/role_assignments?user.id=94edf98909ad47aca0fd0b0702e11163&scope.system=all&role.id=301682fc5c92440f874912eabfed6f70 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:38.117841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10450bf9-b36f-4791-bf0f-039ccc7ec360 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:38.118900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10450bf9-b36f-4791-bf0f-039ccc7ec360 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:38.119404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10450bf9-b36f-4791-bf0f-039ccc7ec360 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:38.121017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 173/335] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:38.126721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5139d6-a1a0-483a-aefe-8ea005882c90 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:38.127580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5139d6-a1a0-483a-aefe-8ea005882c90 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:38.128065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5139d6-a1a0-483a-aefe-8ea005882c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:38.154138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5d5139d6-a1a0-483a-aefe-8ea005882c90 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:38.155699 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5d5139d6-a1a0-483a-aefe-8ea005882c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:38.237302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 163/336] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:38.270852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3ec8f-00e8-461e-a635-a0bbb63c0580 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:38.271487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3ec8f-00e8-461e-a635-a0bbb63c0580 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:38.271969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3ec8f-00e8-461e-a635-a0bbb63c0580 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:38.273627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 174/337] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:38.277604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5139d6-a1a0-483a-aefe-8ea005882c90 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:38.292853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624996-3757-4e11-ab6a-49250ff9b93c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:38.293222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624996-3757-4e11-ab6a-49250ff9b93c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:38.293504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624996-3757-4e11-ab6a-49250ff9b93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:38.334127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a9624996-3757-4e11-ab6a-49250ff9b93c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:38.335350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a9624996-3757-4e11-ab6a-49250ff9b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:38.465063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5139d6-a1a0-483a-aefe-8ea005882c90 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:38.503498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 175/338] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:38.508871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:38.534712 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aa3c1-525c-46bf-b150-1b02d7d09e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:38.535531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aa3c1-525c-46bf-b150-1b02d7d09e5f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:38.535804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aa3c1-525c-46bf-b150-1b02d7d09e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:38.536078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aa3c1-525c-46bf-b150-1b02d7d09e5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:38.543661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624996-3757-4e11-ab6a-49250ff9b93c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:38.560795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7aa3c1-525c-46bf-b150-1b02d7d09e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-cb7aa3c1-525c-46bf-b150-1b02d7d09e5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:38.568888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:38.569866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:38.570663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 164/339] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:38.576263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aa3c1-525c-46bf-b150-1b02d7d09e5f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:38.602527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:38.741992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aa3c1-525c-46bf-b150-1b02d7d09e5f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:38.799331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624996-3757-4e11-ab6a-49250ff9b93c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:38.806650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372b7eb-4697-4947-84b7-7a474d329d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:38.807634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372b7eb-4697-4947-84b7-7a474d329d47 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:38.808137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372b7eb-4697-4947-84b7-7a474d329d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:38.809228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372b7eb-4697-4947-84b7-7a474d329d47 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:38.811396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372b7eb-4697-4947-84b7-7a474d329d47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:38.811961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372b7eb-4697-4947-84b7-7a474d329d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:38.813864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372b7eb-4697-4947-84b7-7a474d329d47 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:38.837646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 165/340] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:38.844490 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372b7eb-4697-4947-84b7-7a474d329d47 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:38.883746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca4d56-b830-4b29-be51-3a53ef11219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:38.885792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca4d56-b830-4b29-be51-3a53ef11219f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:38.886957 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca4d56-b830-4b29-be51-3a53ef11219f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:38.887667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca4d56-b830-4b29-be51-3a53ef11219f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:38.913966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca4d56-b830-4b29-be51-3a53ef11219f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:38.920056 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e4ca4d56-b830-4b29-be51-3a53ef11219f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:38.932643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:38.933673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:38.934466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:38.944003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 176/341] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:38.950516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca4d56-b830-4b29-be51-3a53ef11219f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:39.048523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:39.072411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca4d56-b830-4b29-be51-3a53ef11219f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:39.072997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372b7eb-4697-4947-84b7-7a474d329d47 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:39.123436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeb6b0-6cf9-4e51-bf19-ca158c1c3eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:39.124835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeb6b0-6cf9-4e51-bf19-ca158c1c3eec None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:39.125415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeb6b0-6cf9-4e51-bf19-ca158c1c3eec None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:39.125901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeb6b0-6cf9-4e51-bf19-ca158c1c3eec None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:39.126874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeb6b0-6cf9-4e51-bf19-ca158c1c3eec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:39.127388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeb6b0-6cf9-4e51-bf19-ca158c1c3eec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:39.129118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeb6b0-6cf9-4e51-bf19-ca158c1c3eec None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:39.132458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34f8dd-f721-49fd-9636-85cd52f789fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:39.133136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34f8dd-f721-49fd-9636-85cd52f789fe None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:39.133431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34f8dd-f721-49fd-9636-85cd52f789fe None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:39.133715 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34f8dd-f721-49fd-9636-85cd52f789fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:39.140404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 177/342] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:39.165149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeb6b0-6cf9-4e51-bf19-ca158c1c3eec None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:39.234726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34f8dd-f721-49fd-9636-85cd52f789fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:39.236396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34f8dd-f721-49fd-9636-85cd52f789fe None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:39.303234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:39.308098 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeb6b0-6cf9-4e51-bf19-ca158c1c3eec None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:39.394623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e023f-fd7c-454e-b2be-61cbc79bbf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:39.396400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e023f-fd7c-454e-b2be-61cbc79bbf97 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:39.397277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e023f-fd7c-454e-b2be-61cbc79bbf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:39.398066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e023f-fd7c-454e-b2be-61cbc79bbf97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:39.425212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:39.468488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 166/343] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:49:39.507457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34f8dd-f721-49fd-9636-85cd52f789fe None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:39.545824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e023f-fd7c-454e-b2be-61cbc79bbf97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:39.549057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e023f-fd7c-454e-b2be-61cbc79bbf97 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:39.743399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34f8dd-f721-49fd-9636-85cd52f789fe None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:39.743883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 178/344] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:39.753418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e023f-fd7c-454e-b2be-61cbc79bbf97 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:39.801626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818c3ab-4fee-4a10-8ce7-0fd7583821fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:39.802471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818c3ab-4fee-4a10-8ce7-0fd7583821fe None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:39.802763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818c3ab-4fee-4a10-8ce7-0fd7583821fe None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:39.803044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818c3ab-4fee-4a10-8ce7-0fd7583821fe None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:39.804255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818c3ab-4fee-4a10-8ce7-0fd7583821fe None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:39.805755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818c3ab-4fee-4a10-8ce7-0fd7583821fe None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:39.807472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 167/345] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:39 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:39.817298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a53112-8250-4d22-92a4-2c53ba0ccb65 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:39.817298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a53112-8250-4d22-92a4-2c53ba0ccb65 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:39.817298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a53112-8250-4d22-92a4-2c53ba0ccb65 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:39.817298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 168/346] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:39.822711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11ca73-2f0e-48a6-a169-2f1908a5ce28 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:39.823043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11ca73-2f0e-48a6-a169-2f1908a5ce28 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:39.823289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11ca73-2f0e-48a6-a169-2f1908a5ce28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:39.846913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:39.848911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8c11ca73-2f0e-48a6-a169-2f1908a5ce28 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:39.850149 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8c11ca73-2f0e-48a6-a169-2f1908a5ce28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:39.853271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:39.950528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 169/347] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:39.951415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11ca73-2f0e-48a6-a169-2f1908a5ce28 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:39.966600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e023f-fd7c-454e-b2be-61cbc79bbf97 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:40.038945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6582d4-7369-4c49-aff5-13e60dea80fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:40.041073 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6582d4-7369-4c49-aff5-13e60dea80fc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.041768 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6582d4-7369-4c49-aff5-13e60dea80fc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.043910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6582d4-7369-4c49-aff5-13e60dea80fc None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.045611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6582d4-7369-4c49-aff5-13e60dea80fc None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:40.048868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6582d4-7369-4c49-aff5-13e60dea80fc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:40.053413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 179/348] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:39 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:40.054637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6582d4-7369-4c49-aff5-13e60dea80fc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:40.137180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11ca73-2f0e-48a6-a169-2f1908a5ce28 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:40.211183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9a9dd-1452-44ca-86e9-e78b5eef4858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:40.212061 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9a9dd-1452-44ca-86e9-e78b5eef4858 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.212349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9a9dd-1452-44ca-86e9-e78b5eef4858 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.212622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9a9dd-1452-44ca-86e9-e78b5eef4858 None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.219170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9a9dd-1452-44ca-86e9-e78b5eef4858 None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:40.220643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9a9dd-1452-44ca-86e9-e78b5eef4858 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:40.228523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 170/349] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:39 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:40.228523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9a9dd-1452-44ca-86e9-e78b5eef4858 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:40.229598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:40.261424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6582d4-7369-4c49-aff5-13e60dea80fc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:40.276852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:40.316648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b0046-80ae-42e9-983e-88d03e250cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:40.319180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b0046-80ae-42e9-983e-88d03e250cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.319618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b0046-80ae-42e9-983e-88d03e250cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.320053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b0046-80ae-42e9-983e-88d03e250cb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.346761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b0046-80ae-42e9-983e-88d03e250cb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:40.351410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9a9dd-1452-44ca-86e9-e78b5eef4858 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-8a4b0046-80ae-42e9-983e-88d03e250cb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:40.356189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:40.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:40.358047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 180/350] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:40.359810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca0414-ed41-4e20-84e0-18ecf16eeb26 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.360166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca0414-ed41-4e20-84e0-18ecf16eeb26 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.360516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca0414-ed41-4e20-84e0-18ecf16eeb26 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.362158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 181/351] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:40.362960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca0414-ed41-4e20-84e0-18ecf16eeb26 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:40.399787 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14300a5-99a5-43e8-aa71-2c4c80b10e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:40.400678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14300a5-99a5-43e8-aa71-2c4c80b10e88 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.400969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14300a5-99a5-43e8-aa71-2c4c80b10e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.401282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14300a5-99a5-43e8-aa71-2c4c80b10e88 None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.403044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14300a5-99a5-43e8-aa71-2c4c80b10e88 None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:40.404511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14300a5-99a5-43e8-aa71-2c4c80b10e88 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:40.406410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 171/352] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:40 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:40.406997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14300a5-99a5-43e8-aa71-2c4c80b10e88 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:40.489823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:40.499895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca0414-ed41-4e20-84e0-18ecf16eeb26 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:40.525910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14300a5-99a5-43e8-aa71-2c4c80b10e88 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:40.539242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcfeaf-a963-4c87-ab16-44828199b2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:40.540667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcfeaf-a963-4c87-ab16-44828199b2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.540667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcfeaf-a963-4c87-ab16-44828199b2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.540667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcfeaf-a963-4c87-ab16-44828199b2a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.541291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcfeaf-a963-4c87-ab16-44828199b2a1 None admin] RBAC: Adding query filter params (group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=26049e1a44a94f1ab013063cd9767d44) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:40.542404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcfeaf-a963-4c87-ab16-44828199b2a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:40.544677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcfeaf-a963-4c87-ab16-44828199b2a1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:40.550717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 182/353] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:40 2022] GET /identity/v3/role_assignments?group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=301682fc5c92440f874912eabfed6f70 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:40.551860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfb103-92e7-4a1e-a953-c81d5f0e12f3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.552123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfb103-92e7-4a1e-a953-c81d5f0e12f3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.552398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfb103-92e7-4a1e-a953-c81d5f0e12f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.566881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1614b-2e31-4be9-a3cf-1ea2a9356c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:40.567652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1614b-2e31-4be9-a3cf-1ea2a9356c0b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.568095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1614b-2e31-4be9-a3cf-1ea2a9356c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.568327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1614b-2e31-4be9-a3cf-1ea2a9356c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.569230 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1614b-2e31-4be9-a3cf-1ea2a9356c0b None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:40.569555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1614b-2e31-4be9-a3cf-1ea2a9356c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:40.571009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1614b-2e31-4be9-a3cf-1ea2a9356c0b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:40.572563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-56cfb103-92e7-4a1e-a953-c81d5f0e12f3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:40.573831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-56cfb103-92e7-4a1e-a953-c81d5f0e12f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:40.582255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 172/354] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:40.584234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff93501-dbc7-463b-975a-ad6921d55780 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.584720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff93501-dbc7-463b-975a-ad6921d55780 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.585236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff93501-dbc7-463b-975a-ad6921d55780 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.591743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 173/355] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:40.593138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff93501-dbc7-463b-975a-ad6921d55780 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:40.612371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:40.661553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 183/356] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:40.663406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7959a-5489-424d-821a-49c48534628d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.663932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7959a-5489-424d-821a-49c48534628d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.664414 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7959a-5489-424d-821a-49c48534628d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.666340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 184/357] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:40.667461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7959a-5489-424d-821a-49c48534628d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:40.743962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff93501-dbc7-463b-975a-ad6921d55780 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:40.789729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33709aa7-d4ca-4c49-9791-f59fc8c817ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:40.790738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33709aa7-d4ca-4c49-9791-f59fc8c817ef None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.791179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33709aa7-d4ca-4c49-9791-f59fc8c817ef None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.791467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33709aa7-d4ca-4c49-9791-f59fc8c817ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.792316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33709aa7-d4ca-4c49-9791-f59fc8c817ef None admin] RBAC: Adding query filter params (group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=26049e1a44a94f1ab013063cd9767d44) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:40.792639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33709aa7-d4ca-4c49-9791-f59fc8c817ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:40.794675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33709aa7-d4ca-4c49-9791-f59fc8c817ef None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:40.796679 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:40.801014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 174/358] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:40 2022] GET /identity/v3/role_assignments?group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=8184510545604582b267e4d1871ae9bf => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:40.802610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e8e0-1739-4046-a7a0-67126261a615 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.802739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7959a-5489-424d-821a-49c48534628d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:40.802820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e8e0-1739-4046-a7a0-67126261a615 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.803001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e8e0-1739-4046-a7a0-67126261a615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.825689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5662e8e0-1739-4046-a7a0-67126261a615 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:40.826863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5662e8e0-1739-4046-a7a0-67126261a615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:40.857783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb6b82-2c93-461a-96e2-139b0e0a8a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:40.858637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb6b82-2c93-461a-96e2-139b0e0a8a32 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.859061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb6b82-2c93-461a-96e2-139b0e0a8a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.859344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb6b82-2c93-461a-96e2-139b0e0a8a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.904121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:40.917670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 175/359] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:40.921903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7eff-2d03-4c23-badd-b188562e2440 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:40.922203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7eff-2d03-4c23-badd-b188562e2440 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:40.922535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7eff-2d03-4c23-badd-b188562e2440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:40.951818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb6b82-2c93-461a-96e2-139b0e0a8a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:40.953340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb6b82-2c93-461a-96e2-139b0e0a8a32 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:40.953682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-de0e7eff-2d03-4c23-badd-b188562e2440 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:40.954903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-de0e7eff-2d03-4c23-badd-b188562e2440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:41.053680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 176/360] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:41.054558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7eff-2d03-4c23-badd-b188562e2440 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.095391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 185/361] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:41.096015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb6b82-2c93-461a-96e2-139b0e0a8a32 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.154989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:41.189270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:41.209445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7eff-2d03-4c23-badd-b188562e2440 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.237932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb6b82-2c93-461a-96e2-139b0e0a8a32 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.261305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afe98c-a436-4850-9d52-72342db9cfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.262447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afe98c-a436-4850-9d52-72342db9cfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.262740 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afe98c-a436-4850-9d52-72342db9cfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.263021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afe98c-a436-4850-9d52-72342db9cfd1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.277816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afe98c-a436-4850-9d52-72342db9cfd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:41.278109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021ffc8-c543-49d3-a91a-77dc435c7fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.278765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021ffc8-c543-49d3-a91a-77dc435c7fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.279051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021ffc8-c543-49d3-a91a-77dc435c7fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.279329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021ffc8-c543-49d3-a91a-77dc435c7fe3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-01afe98c-a436-4850-9d52-72342db9cfd1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:41.286831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:41.288769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 177/362] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:41.290265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afe98c-a436-4850-9d52-72342db9cfd1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.297931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021ffc8-c543-49d3-a91a-77dc435c7fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-0021ffc8-c543-49d3-a91a-77dc435c7fe3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:41.307111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:41.308242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 18:49:41.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:41.311320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 186/363] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:41.313117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021ffc8-c543-49d3-a91a-77dc435c7fe3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.430658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afe98c-a436-4850-9d52-72342db9cfd1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.444802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021ffc8-c543-49d3-a91a-77dc435c7fe3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.454190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:41.474904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:41.479121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f220279-f701-4143-91dc-f65fb37a18d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.481419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f220279-f701-4143-91dc-f65fb37a18d0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.481714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f220279-f701-4143-91dc-f65fb37a18d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.481996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f220279-f701-4143-91dc-f65fb37a18d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.488367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43264ac5-b7fe-4baf-a2e6-d5911c1b5431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.489287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43264ac5-b7fe-4baf-a2e6-d5911c1b5431 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.489575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43264ac5-b7fe-4baf-a2e6-d5911c1b5431 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.490103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43264ac5-b7fe-4baf-a2e6-d5911c1b5431 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.491427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43264ac5-b7fe-4baf-a2e6-d5911c1b5431 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:41.492865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43264ac5-b7fe-4baf-a2e6-d5911c1b5431 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:41.494191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f220279-f701-4143-91dc-f65fb37a18d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:41.494915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 187/364] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:41.495730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43264ac5-b7fe-4baf-a2e6-d5911c1b5431 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3f220279-f701-4143-91dc-f65fb37a18d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:41.502180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:41.503477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:41.504457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 178/365] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:41.505304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f220279-f701-4143-91dc-f65fb37a18d0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.617690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f220279-f701-4143-91dc-f65fb37a18d0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.618543 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43264ac5-b7fe-4baf-a2e6-d5911c1b5431 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.657021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2225b-0ad9-4d4b-9b98-96a7506987fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.657459 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253be177-6e91-4d16-b172-d92e9f751ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.657730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2225b-0ad9-4d4b-9b98-96a7506987fb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.658019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2225b-0ad9-4d4b-9b98-96a7506987fb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.658183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be177-6e91-4d16-b172-d92e9f751ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.658328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2225b-0ad9-4d4b-9b98-96a7506987fb None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.658480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be177-6e91-4d16-b172-d92e9f751ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.658623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be177-6e91-4d16-b172-d92e9f751ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.659051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2225b-0ad9-4d4b-9b98-96a7506987fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:41.659335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253be177-6e91-4d16-b172-d92e9f751ea0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:41.659488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2225b-0ad9-4d4b-9b98-96a7506987fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:41.659639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253be177-6e91-4d16-b172-d92e9f751ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:41.660786 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2225b-0ad9-4d4b-9b98-96a7506987fb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:41.661022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253be177-6e91-4d16-b172-d92e9f751ea0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:41.670079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 188/366] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:41.670926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2225b-0ad9-4d4b-9b98-96a7506987fb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.671125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 179/367] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:41.672090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253be177-6e91-4d16-b172-d92e9f751ea0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.718628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:41.732251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:41.782202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253be177-6e91-4d16-b172-d92e9f751ea0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.783976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2225b-0ad9-4d4b-9b98-96a7506987fb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.820536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181b78d-ddde-42f2-8a3f-b8f04d3032dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.821326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181b78d-ddde-42f2-8a3f-b8f04d3032dd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.821731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181b78d-ddde-42f2-8a3f-b8f04d3032dd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.822018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181b78d-ddde-42f2-8a3f-b8f04d3032dd None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.822782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181b78d-ddde-42f2-8a3f-b8f04d3032dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:41.822948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f9bc-d34e-486e-8586-f8c657df45ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.823143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181b78d-ddde-42f2-8a3f-b8f04d3032dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:41.823654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f9bc-d34e-486e-8586-f8c657df45ac None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.823944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f9bc-d34e-486e-8586-f8c657df45ac None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.824806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f9bc-d34e-486e-8586-f8c657df45ac None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.825038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181b78d-ddde-42f2-8a3f-b8f04d3032dd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:41.826030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f9bc-d34e-486e-8586-f8c657df45ac None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:41.827474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f9bc-d34e-486e-8586-f8c657df45ac None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:41.829150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 189/368] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:41.829928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f9bc-d34e-486e-8586-f8c657df45ac None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.834576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 180/369] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:41.835417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181b78d-ddde-42f2-8a3f-b8f04d3032dd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:41.939760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f9bc-d34e-486e-8586-f8c657df45ac None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.945412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181b78d-ddde-42f2-8a3f-b8f04d3032dd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:41.946936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:41.966301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:41.980469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db838-ec7b-4fbb-ad5f-f78dda3fa679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.981296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db838-ec7b-4fbb-ad5f-f78dda3fa679 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.981773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db838-ec7b-4fbb-ad5f-f78dda3fa679 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.982063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db838-ec7b-4fbb-ad5f-f78dda3fa679 None admin] PATH_INFO: `/v3/users/c6528ca55c914dee98277eb6760f1cdd` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.982841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ee334-9bfe-4ecd-9dfe-65c377a18ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:41.983327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30db838-ec7b-4fbb-ad5f-f78dda3fa679 None admin] RBAC: Authorizing `identity:get_user(user_id=c6528ca55c914dee98277eb6760f1cdd)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:41.983514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ee334-9bfe-4ecd-9dfe-65c377a18ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:41.983766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ee334-9bfe-4ecd-9dfe-65c377a18ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:41.984043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ee334-9bfe-4ecd-9dfe-65c377a18ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:41.984877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30db838-ec7b-4fbb-ad5f-f78dda3fa679 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:41.986914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 190/370] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/users/c6528ca55c914dee98277eb6760f1cdd => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:41.987863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db838-ec7b-4fbb-ad5f-f78dda3fa679 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:42.056231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ee334-9bfe-4ecd-9dfe-65c377a18ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:42.057758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ee334-9bfe-4ecd-9dfe-65c377a18ce4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:42.104154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db838-ec7b-4fbb-ad5f-f78dda3fa679 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:42.145478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1bc29-ea3f-4011-a2a6-8e5354c99e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:42.146177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1bc29-ea3f-4011-a2a6-8e5354c99e9e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:42.146422 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1bc29-ea3f-4011-a2a6-8e5354c99e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:42.146754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1bc29-ea3f-4011-a2a6-8e5354c99e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:42.147601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1bc29-ea3f-4011-a2a6-8e5354c99e9e None admin] RBAC: Adding query filter params (group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=45204248d1df42a29cab699e5a889efe) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:42.147919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1bc29-ea3f-4011-a2a6-8e5354c99e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:42.150363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1bc29-ea3f-4011-a2a6-8e5354c99e9e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:42.156795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 191/371] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/role_assignments?group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=301682fc5c92440f874912eabfed6f70 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:42.157648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1bc29-ea3f-4011-a2a6-8e5354c99e9e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:42.195156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 181/372] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:42.196195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ee334-9bfe-4ecd-9dfe-65c377a18ce4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:42.198929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:42.246813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:42.328283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1bc29-ea3f-4011-a2a6-8e5354c99e9e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:42.391129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced1a8e-1eb1-479b-a0e9-210df3756e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:42.391795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced1a8e-1eb1-479b-a0e9-210df3756e42 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:42.392088 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced1a8e-1eb1-479b-a0e9-210df3756e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:42.392379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced1a8e-1eb1-479b-a0e9-210df3756e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:42.415253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ee334-9bfe-4ecd-9dfe-65c377a18ce4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:42.467422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aca70d-65c4-4565-938a-38aa91a30704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:42.468101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca70d-65c4-4565-938a-38aa91a30704 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:42.468388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca70d-65c4-4565-938a-38aa91a30704 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:42.468672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca70d-65c4-4565-938a-38aa91a30704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:42.469757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aca70d-65c4-4565-938a-38aa91a30704 None admin] RBAC: Adding query filter params (role.id=f216586d507240c9b31a20c99b9bd21a, user.id=c6528ca55c914dee98277eb6760f1cdd, scope.system=all) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:42.470083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aca70d-65c4-4565-938a-38aa91a30704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:42.471535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aca70d-65c4-4565-938a-38aa91a30704 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:42.479320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced1a8e-1eb1-479b-a0e9-210df3756e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:42.481360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced1a8e-1eb1-479b-a0e9-210df3756e42 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:42.494958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 182/373] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Jun 21 18:49:42 2022] GET /identity/v3/role_assignments?user.id=c6528ca55c914dee98277eb6760f1cdd&scope.system=all&role.id=f216586d507240c9b31a20c99b9bd21a => generated 510 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:42.495819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aca70d-65c4-4565-938a-38aa91a30704 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:42.541299 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:42.622032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 192/374] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:42.634086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced1a8e-1eb1-479b-a0e9-210df3756e42 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:42.654231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:42.727565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aca70d-65c4-4565-938a-38aa91a30704 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:42.793959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f015c3f-dcf7-4344-af1f-0fd9a797bde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:42.795027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f015c3f-dcf7-4344-af1f-0fd9a797bde3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:42.795412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f015c3f-dcf7-4344-af1f-0fd9a797bde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:42.795796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f015c3f-dcf7-4344-af1f-0fd9a797bde3 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:42.797456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f015c3f-dcf7-4344-af1f-0fd9a797bde3 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:42.799158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f015c3f-dcf7-4344-af1f-0fd9a797bde3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:42.801465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 183/375] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:42 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:42.803290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced1a8e-1eb1-479b-a0e9-210df3756e42 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:42.806601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f015c3f-dcf7-4344-af1f-0fd9a797bde3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:42.875412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968047b-1bd2-4cc1-8d99-faf0fb5db879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:42.877763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968047b-1bd2-4cc1-8d99-faf0fb5db879 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:42.877763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968047b-1bd2-4cc1-8d99-faf0fb5db879 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:42.877763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968047b-1bd2-4cc1-8d99-faf0fb5db879 None admin] PATH_INFO: `/v3/projects/8dc534a786d340398da9cb4c7a7e5355` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:42.880250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968047b-1bd2-4cc1-8d99-faf0fb5db879 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc534a786d340398da9cb4c7a7e5355)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:42.882102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968047b-1bd2-4cc1-8d99-faf0fb5db879 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:42.884264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 193/376] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:42 2022] GET /identity/v3/projects/8dc534a786d340398da9cb4c7a7e5355 => generated 344 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:42.890936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968047b-1bd2-4cc1-8d99-faf0fb5db879 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:42.933269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:42.969616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f015c3f-dcf7-4344-af1f-0fd9a797bde3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:43.007228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:43.011185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba5d6f-7abe-4c36-99fb-d4817caaf238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:43.013250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba5d6f-7abe-4c36-99fb-d4817caaf238 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.013790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba5d6f-7abe-4c36-99fb-d4817caaf238 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.014293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba5d6f-7abe-4c36-99fb-d4817caaf238 None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.015762 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba5d6f-7abe-4c36-99fb-d4817caaf238 None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:43.017607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba5d6f-7abe-4c36-99fb-d4817caaf238 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:43.019788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 184/377] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:42 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:43.025744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba5d6f-7abe-4c36-99fb-d4817caaf238 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:43.093517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968047b-1bd2-4cc1-8d99-faf0fb5db879 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:43.132316 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9cbe4-cacf-41e7-bbc7-bfadfcf98d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:43.133577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9cbe4-cacf-41e7-bbc7-bfadfcf98d6c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.134101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9cbe4-cacf-41e7-bbc7-bfadfcf98d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.134687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9cbe4-cacf-41e7-bbc7-bfadfcf98d6c None admin] PATH_INFO: `/v3/groups/a6dfb9be1a354ef084dbe6001defd08c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.142389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9cbe4-cacf-41e7-bbc7-bfadfcf98d6c None admin] RBAC: Authorizing `identity:get_group(group_id=a6dfb9be1a354ef084dbe6001defd08c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:43.144243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9cbe4-cacf-41e7-bbc7-bfadfcf98d6c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:43.146416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 194/378] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:42 2022] GET /identity/v3/groups/a6dfb9be1a354ef084dbe6001defd08c => generated 229 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:43.147905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247027ed-cc6d-4aa8-b5af-aa5422183316 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.148556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247027ed-cc6d-4aa8-b5af-aa5422183316 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.149046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247027ed-cc6d-4aa8-b5af-aa5422183316 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.150928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 195/379] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:43.152946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247027ed-cc6d-4aa8-b5af-aa5422183316 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:43.160142 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba5d6f-7abe-4c36-99fb-d4817caaf238 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:43.196860 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e96d9-7df4-494b-b3e6-f786d9713ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:43.197536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e96d9-7df4-494b-b3e6-f786d9713ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.197898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e96d9-7df4-494b-b3e6-f786d9713ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.198186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e96d9-7df4-494b-b3e6-f786d9713ef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.199566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e96d9-7df4-494b-b3e6-f786d9713ef8 None admin] RBAC: Adding query filter params (group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=45204248d1df42a29cab699e5a889efe) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:43.199894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e96d9-7df4-494b-b3e6-f786d9713ef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:43.202439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e96d9-7df4-494b-b3e6-f786d9713ef8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:43.208982 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 185/380] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:43 2022] GET /identity/v3/role_assignments?group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=8184510545604582b267e4d1871ae9bf => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:43.210562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e4e67-e913-458b-a255-760dee575081 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.210562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e4e67-e913-458b-a255-760dee575081 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.210844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e4e67-e913-458b-a255-760dee575081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.229769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-478e4e67-e913-458b-a255-760dee575081 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:43.231083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-478e4e67-e913-458b-a255-760dee575081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:43.256852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:43.278974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247027ed-cc6d-4aa8-b5af-aa5422183316 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:43.300448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:43.316949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 186/381] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:43.328514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31897be0-a3c1-438e-b1b9-6565ded9494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:43.329817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31897be0-a3c1-438e-b1b9-6565ded9494a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.330346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31897be0-a3c1-438e-b1b9-6565ded9494a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.330844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31897be0-a3c1-438e-b1b9-6565ded9494a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.331895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31897be0-a3c1-438e-b1b9-6565ded9494a None admin] RBAC: Adding query filter params (group.id=a6dfb9be1a354ef084dbe6001defd08c, role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=8dc534a786d340398da9cb4c7a7e5355) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:43.332421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31897be0-a3c1-438e-b1b9-6565ded9494a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:43.334101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31897be0-a3c1-438e-b1b9-6565ded9494a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:43.341529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 196/382] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:43 2022] GET /identity/v3/role_assignments?group.id=a6dfb9be1a354ef084dbe6001defd08c&scope.project.id=8dc534a786d340398da9cb4c7a7e5355&role.id=e089a0d6040040c4bcf143a680488e3d => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:43.342967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e4e67-e913-458b-a255-760dee575081 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:43.463952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e4e67-e913-458b-a255-760dee575081 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:43.503785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf81e40-d5d5-4442-b99d-ebd1d54cc318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:43.504662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf81e40-d5d5-4442-b99d-ebd1d54cc318 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.504955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf81e40-d5d5-4442-b99d-ebd1d54cc318 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.505266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf81e40-d5d5-4442-b99d-ebd1d54cc318 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.518738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf81e40-d5d5-4442-b99d-ebd1d54cc318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-7bf81e40-d5d5-4442-b99d-ebd1d54cc318 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:43.527027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:43.528085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:43.528924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 187/383] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:43.534527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31897be0-a3c1-438e-b1b9-6565ded9494a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:43.664024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31897be0-a3c1-438e-b1b9-6565ded9494a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:43.696117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b0718-8151-4354-af52-1381d248c78c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.696748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b0718-8151-4354-af52-1381d248c78c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.697301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b0718-8151-4354-af52-1381d248c78c None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.699104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 188/384] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:43.724488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735198-85a4-49f9-ae00-178028a832f7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.724776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735198-85a4-49f9-ae00-178028a832f7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.724968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735198-85a4-49f9-ae00-178028a832f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.734421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1e25b-f898-44bf-8bb7-da00f66a553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:43.736480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1e25b-f898-44bf-8bb7-da00f66a553f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:43.742028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1e25b-f898-44bf-8bb7-da00f66a553f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:43.742625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1e25b-f898-44bf-8bb7-da00f66a553f None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:43.743622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1e25b-f898-44bf-8bb7-da00f66a553f None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:43.744144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1e25b-f898-44bf-8bb7-da00f66a553f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:43.745792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1e25b-f898-44bf-8bb7-da00f66a553f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:43.757719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-21735198-85a4-49f9-ae00-178028a832f7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:43.759275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-21735198-85a4-49f9-ae00-178028a832f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:43.765614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 197/385] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:43.773981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1e25b-f898-44bf-8bb7-da00f66a553f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:43.809476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:43.865908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:43.893593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 189/386] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:43.925735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21735198-85a4-49f9-ae00-178028a832f7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:43.971047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1e25b-f898-44bf-8bb7-da00f66a553f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:44.029821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85d18f-585d-41f0-9ba9-120ef81a2f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:44.030602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85d18f-585d-41f0-9ba9-120ef81a2f7c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:44.030881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85d18f-585d-41f0-9ba9-120ef81a2f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:44.031157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85d18f-585d-41f0-9ba9-120ef81a2f7c None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:44.033080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85d18f-585d-41f0-9ba9-120ef81a2f7c None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:44.034538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85d18f-585d-41f0-9ba9-120ef81a2f7c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:44.040847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 198/387] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:43 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:44.047215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85d18f-585d-41f0-9ba9-120ef81a2f7c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:44.099763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21735198-85a4-49f9-ae00-178028a832f7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:44.143846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581952f-e085-42c6-9df1-41e114ef9378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:44.144670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581952f-e085-42c6-9df1-41e114ef9378 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:44.144955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581952f-e085-42c6-9df1-41e114ef9378 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:44.145268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581952f-e085-42c6-9df1-41e114ef9378 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:44.169247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581952f-e085-42c6-9df1-41e114ef9378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-1581952f-e085-42c6-9df1-41e114ef9378 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:44.181984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:44.182912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:44.184256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 190/388] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:44.193362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581952f-e085-42c6-9df1-41e114ef9378 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:44.216500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:44.249769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85d18f-585d-41f0-9ba9-120ef81a2f7c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:44.274832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:44.316717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624f2b7-e72d-4fcf-a5af-d317811aa6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:44.318389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624f2b7-e72d-4fcf-a5af-d317811aa6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:44.321376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624f2b7-e72d-4fcf-a5af-d317811aa6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:44.322009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624f2b7-e72d-4fcf-a5af-d317811aa6b3 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:44.337206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624f2b7-e72d-4fcf-a5af-d317811aa6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:44.337206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624f2b7-e72d-4fcf-a5af-d317811aa6b3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:44.337206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 199/389] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:44 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:44.337206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624f2b7-e72d-4fcf-a5af-d317811aa6b3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:44.395170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581952f-e085-42c6-9df1-41e114ef9378 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:44.464676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fe7f8-376d-46bf-98f1-2fc9ecb8bc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:44.465445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe7f8-376d-46bf-98f1-2fc9ecb8bc34 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:44.465733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe7f8-376d-46bf-98f1-2fc9ecb8bc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:44.466048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe7f8-376d-46bf-98f1-2fc9ecb8bc34 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:44.466820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fe7f8-376d-46bf-98f1-2fc9ecb8bc34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:44.467128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fe7f8-376d-46bf-98f1-2fc9ecb8bc34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:44.468532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fe7f8-376d-46bf-98f1-2fc9ecb8bc34 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:44.482446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 191/390] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:44.490211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fe7f8-376d-46bf-98f1-2fc9ecb8bc34 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:44.513502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624f2b7-e72d-4fcf-a5af-d317811aa6b3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:44.550958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef120d2-04fc-4710-b7f0-c183bc5606f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:44.551965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef120d2-04fc-4710-b7f0-c183bc5606f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:44.552256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef120d2-04fc-4710-b7f0-c183bc5606f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:44.552537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef120d2-04fc-4710-b7f0-c183bc5606f2 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:44.554769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef120d2-04fc-4710-b7f0-c183bc5606f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=26049e1a44a94f1ab013063cd9767d44, group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:44.556331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef120d2-04fc-4710-b7f0-c183bc5606f2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:44.579337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 200/391] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jun 21 18:49:44 2022] PUT /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:44.673579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fe7f8-376d-46bf-98f1-2fc9ecb8bc34 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:44.675463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:44.715204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c50f3-9226-4df2-b3d8-3c406e2f49f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:44.716081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c50f3-9226-4df2-b3d8-3c406e2f49f3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:44.716368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c50f3-9226-4df2-b3d8-3c406e2f49f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:44.716648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c50f3-9226-4df2-b3d8-3c406e2f49f3 None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:44.717951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c50f3-9226-4df2-b3d8-3c406e2f49f3 None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:44.719475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c50f3-9226-4df2-b3d8-3c406e2f49f3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:44.721255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 192/392] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:44 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:44.726975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef120d2-04fc-4710-b7f0-c183bc5606f2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:44.744946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:44.788837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f813d-88f3-452b-af72-a90d35adc97f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:44.789207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f813d-88f3-452b-af72-a90d35adc97f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:44.789447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f813d-88f3-452b-af72-a90d35adc97f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:44.791131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 193/393] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:44.802532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fea63d-db40-4235-8c94-b2802a08a023 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:44.802849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fea63d-db40-4235-8c94-b2802a08a023 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:44.803098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fea63d-db40-4235-8c94-b2802a08a023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:44.830025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a4fea63d-db40-4235-8c94-b2802a08a023 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:44.831130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a4fea63d-db40-4235-8c94-b2802a08a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:44.890668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef120d2-04fc-4710-b7f0-c183bc5606f2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:44.929625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 194/394] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:44.933143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69a039-0eb8-4537-b3a9-941e0fee4ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:44.933143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69a039-0eb8-4537-b3a9-941e0fee4ebc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:44.933478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69a039-0eb8-4537-b3a9-941e0fee4ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:44.936406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69a039-0eb8-4537-b3a9-941e0fee4ebc None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:44.936406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69a039-0eb8-4537-b3a9-941e0fee4ebc None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:44.936406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69a039-0eb8-4537-b3a9-941e0fee4ebc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:44.939832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 201/395] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:44 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:44.945495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fea63d-db40-4235-8c94-b2802a08a023 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:44.957764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69a039-0eb8-4537-b3a9-941e0fee4ebc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:45.093557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:45.103347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fea63d-db40-4235-8c94-b2802a08a023 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:45.129556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:45.142818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69a039-0eb8-4537-b3a9-941e0fee4ebc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:45.184152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7459579-7f79-4e98-aa6c-d69fc062f4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:45.184968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7459579-7f79-4e98-aa6c-d69fc062f4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:45.185292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7459579-7f79-4e98-aa6c-d69fc062f4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:45.185577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7459579-7f79-4e98-aa6c-d69fc062f4d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:45.197040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7459579-7f79-4e98-aa6c-d69fc062f4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b7459579-7f79-4e98-aa6c-d69fc062f4d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:45.205706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:45.206567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:45.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:45.208133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 202/396] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:45.215427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7459579-7f79-4e98-aa6c-d69fc062f4d1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:45.218422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ce614-1391-4dfc-a5eb-4ffcdcf6abe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:45.219315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ce614-1391-4dfc-a5eb-4ffcdcf6abe2 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:45.219599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ce614-1391-4dfc-a5eb-4ffcdcf6abe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:45.219875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ce614-1391-4dfc-a5eb-4ffcdcf6abe2 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce/roles/8184510545604582b267e4d1871ae9bf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:45.222157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ce614-1391-4dfc-a5eb-4ffcdcf6abe2 None admin] RBAC: Authorizing `identity:create_grant(project_id=26049e1a44a94f1ab013063cd9767d44, group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role_id=8184510545604582b267e4d1871ae9bf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:45.223615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ce614-1391-4dfc-a5eb-4ffcdcf6abe2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:45.260228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 195/397] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jun 21 18:49:44 2022] PUT /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce/roles/8184510545604582b267e4d1871ae9bf => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:45.349646 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7459579-7f79-4e98-aa6c-d69fc062f4d1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:45.388159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981410ad-8f56-4076-b156-4711315961f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:45.388855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981410ad-8f56-4076-b156-4711315961f4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:45.389173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981410ad-8f56-4076-b156-4711315961f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:45.389463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981410ad-8f56-4076-b156-4711315961f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:45.390233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981410ad-8f56-4076-b156-4711315961f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:45.390546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981410ad-8f56-4076-b156-4711315961f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:45.391983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981410ad-8f56-4076-b156-4711315961f4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:45.401742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 203/398] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:45.405468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:45.408485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ce614-1391-4dfc-a5eb-4ffcdcf6abe2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:45.549851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ce614-1391-4dfc-a5eb-4ffcdcf6abe2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:45.588743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d9c2a-f901-4321-ac9f-28a54e5b2769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:45.589678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d9c2a-f901-4321-ac9f-28a54e5b2769 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:45.589971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d9c2a-f901-4321-ac9f-28a54e5b2769 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:45.590250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d9c2a-f901-4321-ac9f-28a54e5b2769 None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:45.591478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d9c2a-f901-4321-ac9f-28a54e5b2769 None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:45.592996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d9c2a-f901-4321-ac9f-28a54e5b2769 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:45.594910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 196/399] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:45 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:45.600506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981410ad-8f56-4076-b156-4711315961f4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:45.713757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981410ad-8f56-4076-b156-4711315961f4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:45.750226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e336c-4dbc-4f56-a077-33cb2cfc8988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:45.751098 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e336c-4dbc-4f56-a077-33cb2cfc8988 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:45.751387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e336c-4dbc-4f56-a077-33cb2cfc8988 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:45.751665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e336c-4dbc-4f56-a077-33cb2cfc8988 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:45.752887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e336c-4dbc-4f56-a077-33cb2cfc8988 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:45.754388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e336c-4dbc-4f56-a077-33cb2cfc8988 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:45.756427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 204/400] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:45 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:45.762290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d9c2a-f901-4321-ac9f-28a54e5b2769 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:45.794820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:45.882509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d9c2a-f901-4321-ac9f-28a54e5b2769 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:45.933240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb10b5-775b-40a9-b0bb-ffb62fe88639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:45.934174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb10b5-775b-40a9-b0bb-ffb62fe88639 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:45.934464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb10b5-775b-40a9-b0bb-ffb62fe88639 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:45.934746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb10b5-775b-40a9-b0bb-ffb62fe88639 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:45.940608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb10b5-775b-40a9-b0bb-ffb62fe88639 None admin] RBAC: Authorizing `identity:create_grant(project_id=45204248d1df42a29cab699e5a889efe, group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:45.942121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb10b5-775b-40a9-b0bb-ffb62fe88639 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:45.962994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 197/401] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jun 21 18:49:45 2022] PUT /identity/v3/projects/45204248d1df42a29cab699e5a889efe/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:46.190864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d0a6-68d4-460d-8cd7-283b83685df5 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:46.191357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d0a6-68d4-460d-8cd7-283b83685df5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:46.191584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d0a6-68d4-460d-8cd7-283b83685df5 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:46.193255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 205/402] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:46.198851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42a28d-dcb6-4ce8-bf18-b0f4bfd06415 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:46.200048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42a28d-dcb6-4ce8-bf18-b0f4bfd06415 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:46.200553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42a28d-dcb6-4ce8-bf18-b0f4bfd06415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:46.223900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-cd42a28d-dcb6-4ce8-bf18-b0f4bfd06415 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:46.225583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cd42a28d-dcb6-4ce8-bf18-b0f4bfd06415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:46.307672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 198/403] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:46.342996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d0a6-68d4-460d-8cd7-283b83685df5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:46.433992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:46.464303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6d957-79dd-45b6-beaa-951069efbf36 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:46.464832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6d957-79dd-45b6-beaa-951069efbf36 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:46.464832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6d957-79dd-45b6-beaa-951069efbf36 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:46.466237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 199/404] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:46.471678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3f862-3011-4ac9-89e5-badeda1e2014 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:46.471968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3f862-3011-4ac9-89e5-badeda1e2014 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:46.472249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3f862-3011-4ac9-89e5-badeda1e2014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:46.502227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e7a3f862-3011-4ac9-89e5-badeda1e2014 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:46.503819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e7a3f862-3011-4ac9-89e5-badeda1e2014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:46.507802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d0a6-68d4-460d-8cd7-283b83685df5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:46.531972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:46.587739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f53cb-4873-4962-b8fa-a148fc22267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:46.588565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f53cb-4873-4962-b8fa-a148fc22267d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:46.588848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f53cb-4873-4962-b8fa-a148fc22267d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:46.589227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f53cb-4873-4962-b8fa-a148fc22267d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:46.602543 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f53cb-4873-4962-b8fa-a148fc22267d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b18f53cb-4873-4962-b8fa-a148fc22267d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:46.612969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:46.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:46.614699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:46.617172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 206/405] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:46.623436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f53cb-4873-4962-b8fa-a148fc22267d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:46.637515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 200/406] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:49:46.667752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3f862-3011-4ac9-89e5-badeda1e2014 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:46.795502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3f862-3011-4ac9-89e5-badeda1e2014 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:46.841955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c991bb9-67a6-4234-9def-80b8a6b42221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:46.843034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c991bb9-67a6-4234-9def-80b8a6b42221 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:46.844497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c991bb9-67a6-4234-9def-80b8a6b42221 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:46.849557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c991bb9-67a6-4234-9def-80b8a6b42221 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:46.862086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f53cb-4873-4962-b8fa-a148fc22267d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:46.882005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c991bb9-67a6-4234-9def-80b8a6b42221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-7c991bb9-67a6-4234-9def-80b8a6b42221 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:46.890387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:46.891289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:46.892100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:46.893027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:46.895819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 201/407] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:46.905736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c991bb9-67a6-4234-9def-80b8a6b42221 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:46.911386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0454b21-0867-458b-a958-44a1efd8954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:46.912059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0454b21-0867-458b-a958-44a1efd8954c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:46.912349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0454b21-0867-458b-a958-44a1efd8954c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:46.912623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0454b21-0867-458b-a958-44a1efd8954c None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:46.916475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0454b21-0867-458b-a958-44a1efd8954c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:46.916795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0454b21-0867-458b-a958-44a1efd8954c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:46.918758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0454b21-0867-458b-a958-44a1efd8954c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:46.930055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:46.943808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 207/408] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:46.952231 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0454b21-0867-458b-a958-44a1efd8954c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:47.111234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c991bb9-67a6-4234-9def-80b8a6b42221 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:47.156335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c18b7-f7c5-4b3e-8968-5190876e775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:47.157015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c18b7-f7c5-4b3e-8968-5190876e775c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.157330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c18b7-f7c5-4b3e-8968-5190876e775c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.157611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c18b7-f7c5-4b3e-8968-5190876e775c None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.158372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c18b7-f7c5-4b3e-8968-5190876e775c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:47.158684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c18b7-f7c5-4b3e-8968-5190876e775c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:47.160091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c18b7-f7c5-4b3e-8968-5190876e775c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:47.166828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0454b21-0867-458b-a958-44a1efd8954c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:47.182100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 202/409] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:47.205851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c18b7-f7c5-4b3e-8968-5190876e775c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:47.225222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202701e5-2025-4e42-aadb-24c5be707f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:47.226085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202701e5-2025-4e42-aadb-24c5be707f28 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.226414 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202701e5-2025-4e42-aadb-24c5be707f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.226693 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202701e5-2025-4e42-aadb-24c5be707f28 None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.227872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202701e5-2025-4e42-aadb-24c5be707f28 None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:47.229399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202701e5-2025-4e42-aadb-24c5be707f28 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:47.231968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 208/410] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:46 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:47.237727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baf3c1-3d24-4c8d-8b16-da87b46d3bb9 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.238077 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baf3c1-3d24-4c8d-8b16-da87b46d3bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.238361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baf3c1-3d24-4c8d-8b16-da87b46d3bb9 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.240030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 209/411] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:47.241125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baf3c1-3d24-4c8d-8b16-da87b46d3bb9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:47.295698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:47.308544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:47.349290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c18b7-f7c5-4b3e-8968-5190876e775c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:47.369794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baf3c1-3d24-4c8d-8b16-da87b46d3bb9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:47.390665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb29c36-c58d-4516-a28c-d9f06dc8c2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:47.391833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb29c36-c58d-4516-a28c-d9f06dc8c2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.392277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb29c36-c58d-4516-a28c-d9f06dc8c2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.392721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb29c36-c58d-4516-a28c-d9f06dc8c2c8 None admin] PATH_INFO: `/v3/groups/a6dfb9be1a354ef084dbe6001defd08c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.394109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb29c36-c58d-4516-a28c-d9f06dc8c2c8 None admin] RBAC: Authorizing `identity:get_group(group_id=a6dfb9be1a354ef084dbe6001defd08c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:47.395765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb29c36-c58d-4516-a28c-d9f06dc8c2c8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:47.397957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 203/412] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:47 2022] GET /identity/v3/groups/a6dfb9be1a354ef084dbe6001defd08c => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:47.399316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc6f26-caaf-48dd-9098-014a03c5b6ba None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.399741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc6f26-caaf-48dd-9098-014a03c5b6ba None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.400252 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc6f26-caaf-48dd-9098-014a03c5b6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.416432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b60e-8d7f-4a67-899a-23f100f9951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:47.417442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b60e-8d7f-4a67-899a-23f100f9951f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.417789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b60e-8d7f-4a67-899a-23f100f9951f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.418254 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b60e-8d7f-4a67-899a-23f100f9951f None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.419584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2b60e-8d7f-4a67-899a-23f100f9951f None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:47.420058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d5cc6f26-caaf-48dd-9098-014a03c5b6ba None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:47.421340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d5cc6f26-caaf-48dd-9098-014a03c5b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:47.421457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2b60e-8d7f-4a67-899a-23f100f9951f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:47.423060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 210/413] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:47 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:47.423966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b60e-8d7f-4a67-899a-23f100f9951f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:47.507536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 204/414] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:47.508929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc6f26-caaf-48dd-9098-014a03c5b6ba None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:47.559340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b60e-8d7f-4a67-899a-23f100f9951f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:47.585004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:47.596615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:47.599615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73444188-1783-40c8-b5ee-5918ee73be4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:47.600482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73444188-1783-40c8-b5ee-5918ee73be4c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.600768 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73444188-1783-40c8-b5ee-5918ee73be4c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.601043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73444188-1783-40c8-b5ee-5918ee73be4c None admin] PATH_INFO: `/v3/projects/8dc534a786d340398da9cb4c7a7e5355` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.602828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73444188-1783-40c8-b5ee-5918ee73be4c None admin] RBAC: Authorizing `identity:get_project(project_id=8dc534a786d340398da9cb4c7a7e5355)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:47.604264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73444188-1783-40c8-b5ee-5918ee73be4c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:47.606323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 211/415] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:47 2022] GET /identity/v3/projects/8dc534a786d340398da9cb4c7a7e5355 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:47.607102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73444188-1783-40c8-b5ee-5918ee73be4c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:47.642238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc6f26-caaf-48dd-9098-014a03c5b6ba None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:47.683491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfd1d1-07ee-47c6-b4ee-c3877f49005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:47.684707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfd1d1-07ee-47c6-b4ee-c3877f49005f None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.685214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfd1d1-07ee-47c6-b4ee-c3877f49005f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.685683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfd1d1-07ee-47c6-b4ee-c3877f49005f None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce/roles/8184510545604582b267e4d1871ae9bf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.688421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfd1d1-07ee-47c6-b4ee-c3877f49005f None admin] RBAC: Authorizing `identity:create_grant(project_id=45204248d1df42a29cab699e5a889efe, group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role_id=8184510545604582b267e4d1871ae9bf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:47.690109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfd1d1-07ee-47c6-b4ee-c3877f49005f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:47.708854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 205/416] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jun 21 18:49:47 2022] PUT /identity/v3/projects/45204248d1df42a29cab699e5a889efe/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce/roles/8184510545604582b267e4d1871ae9bf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:47.710508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfd1d1-07ee-47c6-b4ee-c3877f49005f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:47.756967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73444188-1783-40c8-b5ee-5918ee73be4c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:47.808295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c887b70-944a-44a9-b21a-258554c241f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:47.809523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c887b70-944a-44a9-b21a-258554c241f7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.810017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c887b70-944a-44a9-b21a-258554c241f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.810500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c887b70-944a-44a9-b21a-258554c241f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.823641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c887b70-944a-44a9-b21a-258554c241f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-1c887b70-944a-44a9-b21a-258554c241f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:47.832676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:47.833715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:47.834513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:47.837038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 212/417] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:47.838653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0411825-564c-46ef-b404-c48b22d4b47f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.839149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0411825-564c-46ef-b404-c48b22d4b47f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.840217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0411825-564c-46ef-b404-c48b22d4b47f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.842215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 213/418] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:47.846061 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfd1d1-07ee-47c6-b4ee-c3877f49005f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:47.849451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd0a72-8b69-4788-8add-48cc308bcc39 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.849977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd0a72-8b69-4788-8add-48cc308bcc39 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.850449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd0a72-8b69-4788-8add-48cc308bcc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.868583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-58cd0a72-8b69-4788-8add-48cc308bcc39 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:47.870232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-58cd0a72-8b69-4788-8add-48cc308bcc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:47.882815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:47.896502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ef9ee-aa21-4b10-ac66-8625b4aa7b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:47.897340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ef9ee-aa21-4b10-ac66-8625b4aa7b3b None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:47.897621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ef9ee-aa21-4b10-ac66-8625b4aa7b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:47.897895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ef9ee-aa21-4b10-ac66-8625b4aa7b3b None admin] PATH_INFO: `/v3/projects/8dc534a786d340398da9cb4c7a7e5355/groups/a6dfb9be1a354ef084dbe6001defd08c/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:47.900625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ef9ee-aa21-4b10-ac66-8625b4aa7b3b None admin] RBAC: Authorizing `identity:create_grant(project_id=8dc534a786d340398da9cb4c7a7e5355, group_id=a6dfb9be1a354ef084dbe6001defd08c, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:47.903107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ef9ee-aa21-4b10-ac66-8625b4aa7b3b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:47.921835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 206/419] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jun 21 18:49:47 2022] PUT /identity/v3/projects/8dc534a786d340398da9cb4c7a7e5355/groups/a6dfb9be1a354ef084dbe6001defd08c/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:49:47.922700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ef9ee-aa21-4b10-ac66-8625b4aa7b3b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:47.941626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:47.972982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 214/420] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:47.998516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd0a72-8b69-4788-8add-48cc308bcc39 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:48.049244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ef9ee-aa21-4b10-ac66-8625b4aa7b3b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:48.092334 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4be34-161b-47b8-ab0a-8560d8a13050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:48.093008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4be34-161b-47b8-ab0a-8560d8a13050 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:48.093313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4be34-161b-47b8-ab0a-8560d8a13050 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:48.093589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4be34-161b-47b8-ab0a-8560d8a13050 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:48.094343 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4be34-161b-47b8-ab0a-8560d8a13050 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:48.094651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4be34-161b-47b8-ab0a-8560d8a13050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:48.097221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4be34-161b-47b8-ab0a-8560d8a13050 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:48.106496 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 207/421] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:48.111979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4be34-161b-47b8-ab0a-8560d8a13050 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:48.149885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd0a72-8b69-4788-8add-48cc308bcc39 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:48.166968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:48.189957 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79381a1a-45a0-4f7f-9986-db98d819269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:48.191064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79381a1a-45a0-4f7f-9986-db98d819269b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:48.191550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79381a1a-45a0-4f7f-9986-db98d819269b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:48.192000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79381a1a-45a0-4f7f-9986-db98d819269b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:48.204350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79381a1a-45a0-4f7f-9986-db98d819269b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-79381a1a-45a0-4f7f-9986-db98d819269b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:48.213628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:48.214628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:48.215421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:48.217869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 215/422] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:48.223389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79381a1a-45a0-4f7f-9986-db98d819269b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:48.226727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4be34-161b-47b8-ab0a-8560d8a13050 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:48.279423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:48.286145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dfd4b-a9bc-4289-88d4-fab27701f9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:48.287093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dfd4b-a9bc-4289-88d4-fab27701f9ed None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:48.287294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dfd4b-a9bc-4289-88d4-fab27701f9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:48.287571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dfd4b-a9bc-4289-88d4-fab27701f9ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:48.347926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79381a1a-45a0-4f7f-9986-db98d819269b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:48.412007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dfd4b-a9bc-4289-88d4-fab27701f9ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:48.413783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dfd4b-a9bc-4289-88d4-fab27701f9ed None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:48.445858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60504240-d569-409d-9d91-a918cab39898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:48.446592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60504240-d569-409d-9d91-a918cab39898 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:48.446824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60504240-d569-409d-9d91-a918cab39898 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:48.447108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60504240-d569-409d-9d91-a918cab39898 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:48.447885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60504240-d569-409d-9d91-a918cab39898 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:48.448197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60504240-d569-409d-9d91-a918cab39898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:48.449676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60504240-d569-409d-9d91-a918cab39898 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:48.475438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 216/423] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:48.480684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:48.482083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60504240-d569-409d-9d91-a918cab39898 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:48.531510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 208/424] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:48.536604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dfd4b-a9bc-4289-88d4-fab27701f9ed None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:48.646997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60504240-d569-409d-9d91-a918cab39898 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:48.649467 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:48.653059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dfd4b-a9bc-4289-88d4-fab27701f9ed None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:48.683933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcb237-ecb4-4ab8-a7e5-9ef059545b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:48.684592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcb237-ecb4-4ab8-a7e5-9ef059545b29 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:48.684881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcb237-ecb4-4ab8-a7e5-9ef059545b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:48.685194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcb237-ecb4-4ab8-a7e5-9ef059545b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:48.693209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e65512-db89-45c0-8122-ac6ccedbe3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:48.694418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e65512-db89-45c0-8122-ac6ccedbe3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:48.694978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e65512-db89-45c0-8122-ac6ccedbe3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:48.695444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e65512-db89-45c0-8122-ac6ccedbe3f3 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:48.696984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e65512-db89-45c0-8122-ac6ccedbe3f3 None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:48.698843 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e65512-db89-45c0-8122-ac6ccedbe3f3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:48.700756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 209/425] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:48 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:48.702476 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4261fd-0bdd-4674-94aa-a2a1b4297961 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:48.702967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4261fd-0bdd-4674-94aa-a2a1b4297961 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:48.703459 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4261fd-0bdd-4674-94aa-a2a1b4297961 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:48.709253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 210/426] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:48.710437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4261fd-0bdd-4674-94aa-a2a1b4297961 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:48.748184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:48.757727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcb237-ecb4-4ab8-a7e5-9ef059545b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:48.759185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcb237-ecb4-4ab8-a7e5-9ef059545b29 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:48.823704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4261fd-0bdd-4674-94aa-a2a1b4297961 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:48.869296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 217/427] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:49:48.869296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78291fc-115f-418d-9085-cef319d614e7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:48.869296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78291fc-115f-418d-9085-cef319d614e7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:48.869296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78291fc-115f-418d-9085-cef319d614e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:48.873944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:48.876461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805296d-2586-4087-addb-669c59896dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:48.877808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805296d-2586-4087-addb-669c59896dca None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:48.878321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805296d-2586-4087-addb-669c59896dca None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:48.878794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805296d-2586-4087-addb-669c59896dca None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:48.881849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805296d-2586-4087-addb-669c59896dca None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:48.883834 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805296d-2586-4087-addb-669c59896dca None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:48.885668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a78291fc-115f-418d-9085-cef319d614e7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:48.886116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 211/428] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:48 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:48.886973 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a78291fc-115f-418d-9085-cef319d614e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:48.887538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805296d-2586-4087-addb-669c59896dca None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:48.972799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 218/429] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:48.972799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78291fc-115f-418d-9085-cef319d614e7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:49.013888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:49.032259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805296d-2586-4087-addb-669c59896dca None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:49.071236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2ba27-1fe3-46b0-b33f-db424aa43788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:49.072474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2ba27-1fe3-46b0-b33f-db424aa43788 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:49.073063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2ba27-1fe3-46b0-b33f-db424aa43788 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:49.073531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2ba27-1fe3-46b0-b33f-db424aa43788 None admin] PATH_INFO: `/v3/projects/26049e1a44a94f1ab013063cd9767d44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:49.075574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2ba27-1fe3-46b0-b33f-db424aa43788 None admin] RBAC: Authorizing `identity:get_project(project_id=26049e1a44a94f1ab013063cd9767d44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:49.077332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2ba27-1fe3-46b0-b33f-db424aa43788 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:49.079337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 212/430] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:48 2022] GET /identity/v3/projects/26049e1a44a94f1ab013063cd9767d44 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:49.080460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2ba27-1fe3-46b0-b33f-db424aa43788 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:49.106803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78291fc-115f-418d-9085-cef319d614e7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:49.154522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a65b58-1ec8-4274-9f3e-424026a67e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:49.156140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a65b58-1ec8-4274-9f3e-424026a67e0a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:49.156655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a65b58-1ec8-4274-9f3e-424026a67e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:49.157072 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a65b58-1ec8-4274-9f3e-424026a67e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:49.158087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a65b58-1ec8-4274-9f3e-424026a67e0a None admin] RBAC: Adding query filter params (group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=26049e1a44a94f1ab013063cd9767d44) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:49.158539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a65b58-1ec8-4274-9f3e-424026a67e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:49.161029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a65b58-1ec8-4274-9f3e-424026a67e0a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:49.163793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:49.171451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 219/431] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:48 2022] GET /identity/v3/role_assignments?group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=301682fc5c92440f874912eabfed6f70 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:49.173061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a65b58-1ec8-4274-9f3e-424026a67e0a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:49.205537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2ba27-1fe3-46b0-b33f-db424aa43788 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:49.247559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1d233-de23-4616-831e-809677e96a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:49.248528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1d233-de23-4616-831e-809677e96a93 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:49.248865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1d233-de23-4616-831e-809677e96a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:49.249184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1d233-de23-4616-831e-809677e96a93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:49.282714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1d233-de23-4616-831e-809677e96a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-e4b1d233-de23-4616-831e-809677e96a93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:49.293992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:49.294971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 18:49:49.295775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:49.301977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 213/432] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:49.308279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1d233-de23-4616-831e-809677e96a93 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:49.359881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:49.368428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a65b58-1ec8-4274-9f3e-424026a67e0a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:49.427763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891b543-dc76-48c7-b40f-9e79852b2526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:49.428813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891b543-dc76-48c7-b40f-9e79852b2526 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:49.429339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891b543-dc76-48c7-b40f-9e79852b2526 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:49.429676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891b543-dc76-48c7-b40f-9e79852b2526 None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:49.431192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891b543-dc76-48c7-b40f-9e79852b2526 None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:49.432765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891b543-dc76-48c7-b40f-9e79852b2526 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:49.436044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 220/433] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:49 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:49.442208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891b543-dc76-48c7-b40f-9e79852b2526 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:49.468771 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1d233-de23-4616-831e-809677e96a93 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:49.507984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d5f0b-f5bd-4c88-a00b-5c086e849c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:49.508563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d5f0b-f5bd-4c88-a00b-5c086e849c98 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:49.508876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d5f0b-f5bd-4c88-a00b-5c086e849c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:49.509163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d5f0b-f5bd-4c88-a00b-5c086e849c98 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:49.509932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d5f0b-f5bd-4c88-a00b-5c086e849c98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:49.510246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d5f0b-f5bd-4c88-a00b-5c086e849c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:49.512214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d5f0b-f5bd-4c88-a00b-5c086e849c98 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:49.523628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 214/434] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:49:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:49.529263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d5f0b-f5bd-4c88-a00b-5c086e849c98 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:49.534196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:49.592004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891b543-dc76-48c7-b40f-9e79852b2526 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:49.653729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fcdae-2770-4d66-936f-1efb4c511253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:49.656917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fcdae-2770-4d66-936f-1efb4c511253 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:49.657438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fcdae-2770-4d66-936f-1efb4c511253 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:49.657897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fcdae-2770-4d66-936f-1efb4c511253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:49.659560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fcdae-2770-4d66-936f-1efb4c511253 None admin] RBAC: Adding query filter params (group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=26049e1a44a94f1ab013063cd9767d44) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:49.660118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fcdae-2770-4d66-936f-1efb4c511253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:49.662833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fcdae-2770-4d66-936f-1efb4c511253 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:49.673884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 221/435] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:49 2022] GET /identity/v3/role_assignments?group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce&scope.project.id=26049e1a44a94f1ab013063cd9767d44&role.id=8184510545604582b267e4d1871ae9bf => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:49.675401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:49.704491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d5f0b-f5bd-4c88-a00b-5c086e849c98 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:49.743776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ede82-212a-4981-942e-577b91ad6d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:49.744450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ede82-212a-4981-942e-577b91ad6d6b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:49.744736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ede82-212a-4981-942e-577b91ad6d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:49.745014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ede82-212a-4981-942e-577b91ad6d6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:49.822607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ede82-212a-4981-942e-577b91ad6d6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:49.824602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ede82-212a-4981-942e-577b91ad6d6b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:49.910601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:49.937829 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 215/436] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:49:49.943794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fcdae-2770-4d66-936f-1efb4c511253 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:50.070799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fcdae-2770-4d66-936f-1efb4c511253 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:50.103060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:50.110170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bf0e0-0ef0-446a-be8b-ef1630a1b1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:50.111046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bf0e0-0ef0-446a-be8b-ef1630a1b1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.111331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bf0e0-0ef0-446a-be8b-ef1630a1b1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.111609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bf0e0-0ef0-446a-be8b-ef1630a1b1f3 None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.112835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bf0e0-0ef0-446a-be8b-ef1630a1b1f3 None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:50.114337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bf0e0-0ef0-446a-be8b-ef1630a1b1f3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:50.116257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 222/437] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:49 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:50.121616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ede82-212a-4981-942e-577b91ad6d6b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:50.189041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192f478-d659-4596-adfe-c8188ae2f2c5 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.189719 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192f478-d659-4596-adfe-c8188ae2f2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.190220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192f478-d659-4596-adfe-c8188ae2f2c5 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.192052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 223/438] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:49:50.204127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773395c7-a01b-4248-b918-ba5be2f3f3fb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.204127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773395c7-a01b-4248-b918-ba5be2f3f3fb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.204127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773395c7-a01b-4248-b918-ba5be2f3f3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.230083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-773395c7-a01b-4248-b918-ba5be2f3f3fb None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:50.231229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-773395c7-a01b-4248-b918-ba5be2f3f3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:50.245014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ede82-212a-4981-942e-577b91ad6d6b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:50.283589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca15c6-9071-4c4e-8464-c6917ac84fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:50.284458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca15c6-9071-4c4e-8464-c6917ac84fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.284743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca15c6-9071-4c4e-8464-c6917ac84fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.285021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca15c6-9071-4c4e-8464-c6917ac84fc8 None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.286958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca15c6-9071-4c4e-8464-c6917ac84fc8 None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:50.288410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca15c6-9071-4c4e-8464-c6917ac84fc8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:50.290251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 216/439] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:50 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:50.295743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca15c6-9071-4c4e-8464-c6917ac84fc8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:50.312258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 224/440] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:50.339665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773395c7-a01b-4248-b918-ba5be2f3f3fb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:50.352879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:50.410287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca15c6-9071-4c4e-8464-c6917ac84fc8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:50.445944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773395c7-a01b-4248-b918-ba5be2f3f3fb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:50.451600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe62159-49ee-409d-a688-5a47bcd07b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:50.452257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62159-49ee-409d-a688-5a47bcd07b73 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.452546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62159-49ee-409d-a688-5a47bcd07b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.452829 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62159-49ee-409d-a688-5a47bcd07b73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.453701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62159-49ee-409d-a688-5a47bcd07b73 None admin] RBAC: Adding query filter params (group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role.id=301682fc5c92440f874912eabfed6f70, scope.project.id=45204248d1df42a29cab699e5a889efe) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:50.454023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62159-49ee-409d-a688-5a47bcd07b73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:50.455465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62159-49ee-409d-a688-5a47bcd07b73 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:50.461445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 217/441] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:50 2022] GET /identity/v3/role_assignments?group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=301682fc5c92440f874912eabfed6f70 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:50.462705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0380638-0624-4867-af86-438f2f7bcc64 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.462998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0380638-0624-4867-af86-438f2f7bcc64 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.463284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0380638-0624-4867-af86-438f2f7bcc64 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.469179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 218/442] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:50.470836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe34a92-d21a-4c56-81aa-bd5d86e7646a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.471129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe34a92-d21a-4c56-81aa-bd5d86e7646a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.471507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe34a92-d21a-4c56-81aa-bd5d86e7646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.475297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:50.485043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981feda-4ab9-40db-b814-251b7e0d8e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:50.485910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981feda-4ab9-40db-b814-251b7e0d8e3b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.486192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981feda-4ab9-40db-b814-251b7e0d8e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.486467 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981feda-4ab9-40db-b814-251b7e0d8e3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.490101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ffe34a92-d21a-4c56-81aa-bd5d86e7646a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:50.491364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ffe34a92-d21a-4c56-81aa-bd5d86e7646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:50.500070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981feda-4ab9-40db-b814-251b7e0d8e3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3981feda-4ab9-40db-b814-251b7e0d8e3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:50.508399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:50.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:50.510462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 225/443] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:50.516280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981feda-4ab9-40db-b814-251b7e0d8e3b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:50.581792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 219/444] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:50.596395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe34a92-d21a-4c56-81aa-bd5d86e7646a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:50.638850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981feda-4ab9-40db-b814-251b7e0d8e3b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:50.642831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:50.684120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d607885-2914-449e-9651-ac04ea7cd4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:50.684803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d607885-2914-449e-9651-ac04ea7cd4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.685132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d607885-2914-449e-9651-ac04ea7cd4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.685393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d607885-2914-449e-9651-ac04ea7cd4b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.686149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d607885-2914-449e-9651-ac04ea7cd4b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:50.686459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d607885-2914-449e-9651-ac04ea7cd4b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:50.689129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d607885-2914-449e-9651-ac04ea7cd4b0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:50.697127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 226/445] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:50.703185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d607885-2914-449e-9651-ac04ea7cd4b0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:50.714974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe34a92-d21a-4c56-81aa-bd5d86e7646a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:50.759089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7ceb2-d837-4835-91f8-e316b43b8160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:50.759089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7ceb2-d837-4835-91f8-e316b43b8160 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.759364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7ceb2-d837-4835-91f8-e316b43b8160 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.759433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7ceb2-d837-4835-91f8-e316b43b8160 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.760316 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:50.773170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7ceb2-d837-4835-91f8-e316b43b8160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-9de7ceb2-d837-4835-91f8-e316b43b8160 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:50.780823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:50.781884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:50.783474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 220/446] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:49:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:50.787837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7ceb2-d837-4835-91f8-e316b43b8160 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:50.821651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d607885-2914-449e-9651-ac04ea7cd4b0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:50.859768 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8554de-a771-4521-8e66-9764d1cd2847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:50.860418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8554de-a771-4521-8e66-9764d1cd2847 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.860697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8554de-a771-4521-8e66-9764d1cd2847 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.860973 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8554de-a771-4521-8e66-9764d1cd2847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.893511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:50.898758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7ceb2-d837-4835-91f8-e316b43b8160 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:50.932477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8554de-a771-4521-8e66-9764d1cd2847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:50.934621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8554de-a771-4521-8e66-9764d1cd2847 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:50.935680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a0e5d-d50b-4af0-a2ac-8badc111d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:50.936353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a0e5d-d50b-4af0-a2ac-8badc111d5da None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:50.936625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a0e5d-d50b-4af0-a2ac-8badc111d5da None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:50.936903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a0e5d-d50b-4af0-a2ac-8badc111d5da None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:50.937946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a0e5d-d50b-4af0-a2ac-8badc111d5da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:50.938255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a0e5d-d50b-4af0-a2ac-8badc111d5da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:50.939620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a0e5d-d50b-4af0-a2ac-8badc111d5da None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:50.947974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 221/447] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:50.953633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a0e5d-d50b-4af0-a2ac-8badc111d5da None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:50.982838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:51.034614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 227/448] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:49:51.040117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8554de-a771-4521-8e66-9764d1cd2847 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:51.060421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a0e5d-d50b-4af0-a2ac-8badc111d5da None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:51.093655 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfa6fa-5cf5-4836-aa8c-d04f0bed4833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:51.094299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfa6fa-5cf5-4836-aa8c-d04f0bed4833 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:51.094587 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfa6fa-5cf5-4836-aa8c-d04f0bed4833 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:51.094866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfa6fa-5cf5-4836-aa8c-d04f0bed4833 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:51.122812 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:51.141739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8554de-a771-4521-8e66-9764d1cd2847 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:51.161918 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfa6fa-5cf5-4836-aa8c-d04f0bed4833 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:51.163371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfa6fa-5cf5-4836-aa8c-d04f0bed4833 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:51.175396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c7d36-4a87-48e6-8ccc-13359c1ccabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:51.176500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c7d36-4a87-48e6-8ccc-13359c1ccabc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:51.176782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c7d36-4a87-48e6-8ccc-13359c1ccabc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:51.177057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c7d36-4a87-48e6-8ccc-13359c1ccabc None admin] PATH_INFO: `/v3/projects/45204248d1df42a29cab699e5a889efe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:51.178381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c7d36-4a87-48e6-8ccc-13359c1ccabc None admin] RBAC: Authorizing `identity:get_project(project_id=45204248d1df42a29cab699e5a889efe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:51.180813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c7d36-4a87-48e6-8ccc-13359c1ccabc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:51.183956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 228/449] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:51 2022] GET /identity/v3/projects/45204248d1df42a29cab699e5a889efe => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:51.191479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c7d36-4a87-48e6-8ccc-13359c1ccabc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:51.209807 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:51.263102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 222/450] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:49:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 18:49:51.268662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfa6fa-5cf5-4836-aa8c-d04f0bed4833 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:51.291595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c7d36-4a87-48e6-8ccc-13359c1ccabc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:51.327238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:51.328920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05b828-ce90-4e6f-a1bb-8c7b27f1601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:51.329753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05b828-ce90-4e6f-a1bb-8c7b27f1601e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:51.330045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05b828-ce90-4e6f-a1bb-8c7b27f1601e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:51.330333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05b828-ce90-4e6f-a1bb-8c7b27f1601e None admin] PATH_INFO: `/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:51.331542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05b828-ce90-4e6f-a1bb-8c7b27f1601e None admin] RBAC: Authorizing `identity:get_group(group_id=c32f0ba7b2ea461eb0eeaf4bee8d8fce)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:51.332989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05b828-ce90-4e6f-a1bb-8c7b27f1601e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:51.334633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 229/451] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:51 2022] GET /identity/v3/groups/c32f0ba7b2ea461eb0eeaf4bee8d8fce => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:51.340931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05b828-ce90-4e6f-a1bb-8c7b27f1601e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:51.370090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfa6fa-5cf5-4836-aa8c-d04f0bed4833 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:51.403851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ee3ac-9b3c-4798-9d6d-2b362bd3d10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:51.404692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ee3ac-9b3c-4798-9d6d-2b362bd3d10d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:51.404969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ee3ac-9b3c-4798-9d6d-2b362bd3d10d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:51.405274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ee3ac-9b3c-4798-9d6d-2b362bd3d10d None admin] PATH_INFO: `/v3/projects/8dc534a786d340398da9cb4c7a7e5355` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:51.406456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ee3ac-9b3c-4798-9d6d-2b362bd3d10d None admin] RBAC: Authorizing `identity:get_project(project_id=8dc534a786d340398da9cb4c7a7e5355)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:51.407894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ee3ac-9b3c-4798-9d6d-2b362bd3d10d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:51.409612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 223/452] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:49:51 2022] GET /identity/v3/projects/8dc534a786d340398da9cb4c7a7e5355 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:51.414940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ee3ac-9b3c-4798-9d6d-2b362bd3d10d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:51.426462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:51.443061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05b828-ce90-4e6f-a1bb-8c7b27f1601e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:51.478027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a14789-3a97-45c0-8951-124364529426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:51.478726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a14789-3a97-45c0-8951-124364529426 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:51.479016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a14789-3a97-45c0-8951-124364529426 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:51.479298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a14789-3a97-45c0-8951-124364529426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:51.480762 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a14789-3a97-45c0-8951-124364529426 None admin] RBAC: Adding query filter params (group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce, role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=45204248d1df42a29cab699e5a889efe) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:51.481103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a14789-3a97-45c0-8951-124364529426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:51.482583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a14789-3a97-45c0-8951-124364529426 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:51.488917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 230/453] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:51 2022] GET /identity/v3/role_assignments?group.id=c32f0ba7b2ea461eb0eeaf4bee8d8fce&scope.project.id=45204248d1df42a29cab699e5a889efe&role.id=8184510545604582b267e4d1871ae9bf => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:51.530382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ee3ac-9b3c-4798-9d6d-2b362bd3d10d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:51.562139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:51.571898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccdf3e-e91b-4a12-aa6f-c62bb2041de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:51.572777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccdf3e-e91b-4a12-aa6f-c62bb2041de1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:51.573065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccdf3e-e91b-4a12-aa6f-c62bb2041de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:51.573383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccdf3e-e91b-4a12-aa6f-c62bb2041de1 None admin] PATH_INFO: `/v3/groups/a6dfb9be1a354ef084dbe6001defd08c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:51.574729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccdf3e-e91b-4a12-aa6f-c62bb2041de1 None admin] RBAC: Authorizing `identity:get_group(group_id=a6dfb9be1a354ef084dbe6001defd08c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:51.576203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccdf3e-e91b-4a12-aa6f-c62bb2041de1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:51.577931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 224/454] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jun 21 18:49:51 2022] GET /identity/v3/groups/a6dfb9be1a354ef084dbe6001defd08c => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:51.583980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a14789-3a97-45c0-8951-124364529426 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:51.689269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a14789-3a97-45c0-8951-124364529426 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:51.724320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9bd7f-f9f7-4168-9f9d-eeb0c6f5fe9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:51.724906 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9bd7f-f9f7-4168-9f9d-eeb0c6f5fe9c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:51.725225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9bd7f-f9f7-4168-9f9d-eeb0c6f5fe9c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:51.725508 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9bd7f-f9f7-4168-9f9d-eeb0c6f5fe9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:51.726353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9bd7f-f9f7-4168-9f9d-eeb0c6f5fe9c None admin] RBAC: Adding query filter params (group.id=a6dfb9be1a354ef084dbe6001defd08c, role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=8dc534a786d340398da9cb4c7a7e5355) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:51.726672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9bd7f-f9f7-4168-9f9d-eeb0c6f5fe9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:51.728123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9bd7f-f9f7-4168-9f9d-eeb0c6f5fe9c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:51.734669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 231/455] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jun 21 18:49:51 2022] GET /identity/v3/role_assignments?group.id=a6dfb9be1a354ef084dbe6001defd08c&scope.project.id=8dc534a786d340398da9cb4c7a7e5355&role.id=e089a0d6040040c4bcf143a680488e3d => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:54.393482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c37e1-b029-4b6e-9396-c9ce05671a54 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:54.393996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c37e1-b029-4b6e-9396-c9ce05671a54 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:54.394070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c37e1-b029-4b6e-9396-c9ce05671a54 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:54.395694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 225/456] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:54.403195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7fa53-4aa7-4566-8f0e-5750f461ee02 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:54.403683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7fa53-4aa7-4566-8f0e-5750f461ee02 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:54.403756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7fa53-4aa7-4566-8f0e-5750f461ee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:54.434212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b1e7fa53-4aa7-4566-8f0e-5750f461ee02 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:54.435435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b1e7fa53-4aa7-4566-8f0e-5750f461ee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:54.447034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:54.527856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95057-e26e-4573-a563-d069b454d9e0 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:54.529446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95057-e26e-4573-a563-d069b454d9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:54.529937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95057-e26e-4573-a563-d069b454d9e0 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:54.531823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 226/457] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:54.538057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e7c91-5062-4188-bad3-d59c381b2713 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:54.539376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e7c91-5062-4188-bad3-d59c381b2713 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:54.541370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e7c91-5062-4188-bad3-d59c381b2713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:54.542339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 232/458] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:54.543857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116905fb-8c09-427a-bf0a-533ff26e526b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:54.544150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116905fb-8c09-427a-bf0a-533ff26e526b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:54.544434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116905fb-8c09-427a-bf0a-533ff26e526b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:54.546074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 233/459] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:54.551860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917b4e0-10ab-4355-b3f0-196bfb19cc27 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:54.552144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917b4e0-10ab-4355-b3f0-196bfb19cc27 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:54.552425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917b4e0-10ab-4355-b3f0-196bfb19cc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:54.570776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-088e7c91-5062-4188-bad3-d59c381b2713 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:54.572351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-088e7c91-5062-4188-bad3-d59c381b2713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:54.583423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5917b4e0-10ab-4355-b3f0-196bfb19cc27 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:54.583850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5917b4e0-10ab-4355-b3f0-196bfb19cc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:54.650781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 227/460] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:54.652028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e7c91-5062-4188-bad3-d59c381b2713 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:54.671713 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 234/461] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:54.686662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917b4e0-10ab-4355-b3f0-196bfb19cc27 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:54.763754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e7c91-5062-4188-bad3-d59c381b2713 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:54.770115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:54.793439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:54.800805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917b4e0-10ab-4355-b3f0-196bfb19cc27 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:54.807159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3c90a-84cf-48ed-91e6-8166ae13b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:54.808263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c90a-84cf-48ed-91e6-8166ae13b42a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:54.809296 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c90a-84cf-48ed-91e6-8166ae13b42a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:54.809779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c90a-84cf-48ed-91e6-8166ae13b42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:54.825119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3c90a-84cf-48ed-91e6-8166ae13b42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:54.827369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3c90a-84cf-48ed-91e6-8166ae13b42a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:54.839315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f9ee1-f4f1-4b5b-8db9-ceb65545316d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:54.840440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f9ee1-f4f1-4b5b-8db9-ceb65545316d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:54.840928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f9ee1-f4f1-4b5b-8db9-ceb65545316d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:54.841452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f9ee1-f4f1-4b5b-8db9-ceb65545316d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d1f3c90a-84cf-48ed-91e6-8166ae13b42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:49:54.850012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:54.850912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:54.851751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:49:54.851751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:49:54.851751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:54.851751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:54.851751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 228/462] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:54.852644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3c90a-84cf-48ed-91e6-8166ae13b42a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:54.858151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f9ee1-f4f1-4b5b-8db9-ceb65545316d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:54.859915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f9ee1-f4f1-4b5b-8db9-ceb65545316d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-622f9ee1-f4f1-4b5b-8db9-ceb65545316d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:49:54.879444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:54.880369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:54.881224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:49:54.881224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:49:54.881224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:54.881224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:54.890743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 235/463] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:54.892200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f9ee1-f4f1-4b5b-8db9-ceb65545316d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:54.975509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3c90a-84cf-48ed-91e6-8166ae13b42a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:55.012815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd5b98-cabc-49de-af1f-45fa30f5a57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:55.014076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd5b98-cabc-49de-af1f-45fa30f5a57f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.014600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd5b98-cabc-49de-af1f-45fa30f5a57f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.015098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd5b98-cabc-49de-af1f-45fa30f5a57f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.027407 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd5b98-cabc-49de-af1f-45fa30f5a57f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:55.029352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd5b98-cabc-49de-af1f-45fa30f5a57f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:55.036845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f9ee1-f4f1-4b5b-8db9-ceb65545316d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3dcd5b98-cabc-49de-af1f-45fa30f5a57f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:49:55.050564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:55.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:55.052385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:49:55.052385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:49:55.052385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:55.052385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:55.054447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 229/464] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:55.055582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd5b98-cabc-49de-af1f-45fa30f5a57f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:55.070336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:55.080672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d48448-a957-4b41-a1dd-9aeee38fa1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:55.081420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d48448-a957-4b41-a1dd-9aeee38fa1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.081712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d48448-a957-4b41-a1dd-9aeee38fa1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.081991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d48448-a957-4b41-a1dd-9aeee38fa1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.082744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d48448-a957-4b41-a1dd-9aeee38fa1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:55.083032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d48448-a957-4b41-a1dd-9aeee38fa1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:55.084517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d48448-a957-4b41-a1dd-9aeee38fa1e5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:55.091028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:55.099004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 236/465] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:55.099851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d48448-a957-4b41-a1dd-9aeee38fa1e5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:55.168040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd5b98-cabc-49de-af1f-45fa30f5a57f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:55.202265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b652b-e2c4-485c-9c44-e0a865d7fe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:55.202940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b652b-e2c4-485c-9c44-e0a865d7fe4d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.203229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b652b-e2c4-485c-9c44-e0a865d7fe4d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.203507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b652b-e2c4-485c-9c44-e0a865d7fe4d None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.204399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b652b-e2c4-485c-9c44-e0a865d7fe4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:55.204717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b652b-e2c4-485c-9c44-e0a865d7fe4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:55.206162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b652b-e2c4-485c-9c44-e0a865d7fe4d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:55.216734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d48448-a957-4b41-a1dd-9aeee38fa1e5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:55.224392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 230/466] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:55.230782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4573970-0774-4bbd-bd8f-8a0081423632 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.231076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4573970-0774-4bbd-bd8f-8a0081423632 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.231363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4573970-0774-4bbd-bd8f-8a0081423632 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.232721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 231/467] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:55.233445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4573970-0774-4bbd-bd8f-8a0081423632 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:55.256872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b5ad2-3284-46f0-afcd-7623d934e833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:55.257783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b5ad2-3284-46f0-afcd-7623d934e833 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.258241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b5ad2-3284-46f0-afcd-7623d934e833 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.258711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b5ad2-3284-46f0-afcd-7623d934e833 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.259755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b5ad2-3284-46f0-afcd-7623d934e833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:55.260219 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b5ad2-3284-46f0-afcd-7623d934e833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:55.261900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b5ad2-3284-46f0-afcd-7623d934e833 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:55.276064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 237/468] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:55.277780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b5ad2-3284-46f0-afcd-7623d934e833 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:55.329806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:55.340044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:55.352576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4573970-0774-4bbd-bd8f-8a0081423632 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:55.387004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b5ad2-3284-46f0-afcd-7623d934e833 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:55.388848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6a4bc-3266-4d77-9324-7f1ac1d49be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:55.389561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6a4bc-3266-4d77-9324-7f1ac1d49be6 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.389889 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6a4bc-3266-4d77-9324-7f1ac1d49be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.390174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6a4bc-3266-4d77-9324-7f1ac1d49be6 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.390896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6a4bc-3266-4d77-9324-7f1ac1d49be6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:55.392390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6a4bc-3266-4d77-9324-7f1ac1d49be6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:55.425135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 232/469] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:55 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:55.425135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf2b62-646e-4b8a-af42-615349f1bbef None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.425135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf2b62-646e-4b8a-af42-615349f1bbef None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.425135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf2b62-646e-4b8a-af42-615349f1bbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.438469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea5cab-d944-4a42-9675-ec2a994bcac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:55.439310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea5cab-d944-4a42-9675-ec2a994bcac1 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.439658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea5cab-d944-4a42-9675-ec2a994bcac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.440100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea5cab-d944-4a42-9675-ec2a994bcac1 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.441058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea5cab-d944-4a42-9675-ec2a994bcac1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:55.442783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea5cab-d944-4a42-9675-ec2a994bcac1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:55.450372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e9cf2b62-646e-4b8a-af42-615349f1bbef None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:55.452000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e9cf2b62-646e-4b8a-af42-615349f1bbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:55.466545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 238/470] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:55 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:55.467820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea5cab-d944-4a42-9675-ec2a994bcac1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:55.565656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 233/471] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:55.567647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e398045-0fc7-4b20-bc83-60eb1feec4c4 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.568144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e398045-0fc7-4b20-bc83-60eb1feec4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.568562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e398045-0fc7-4b20-bc83-60eb1feec4c4 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.575058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 234/472] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:55.580432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c09f0-4413-4ca3-85a8-a9a1408b3370 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.581048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c09f0-4413-4ca3-85a8-a9a1408b3370 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.581735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c09f0-4413-4ca3-85a8-a9a1408b3370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.608286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea5cab-d944-4a42-9675-ec2a994bcac1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:55.610272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:55.615825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-cf8c09f0-4413-4ca3-85a8-a9a1408b3370 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:55.617224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cf8c09f0-4413-4ca3-85a8-a9a1408b3370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:55.654771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f31b9-1621-4623-979f-8f1f4cfdb698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:55.655701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f31b9-1621-4623-979f-8f1f4cfdb698 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.656137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f31b9-1621-4623-979f-8f1f4cfdb698 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.656590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f31b9-1621-4623-979f-8f1f4cfdb698 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.657513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f31b9-1621-4623-979f-8f1f4cfdb698 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:55.659210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f31b9-1621-4623-979f-8f1f4cfdb698 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:55.688515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 239/473] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:55 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:55.689611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f31b9-1621-4623-979f-8f1f4cfdb698 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:55.706176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 235/474] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:55.729464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c09f0-4413-4ca3-85a8-a9a1408b3370 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:55.740468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:55.856196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f31b9-1621-4623-979f-8f1f4cfdb698 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:55.889519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c09f0-4413-4ca3-85a8-a9a1408b3370 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:55.901742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9e60f-4a7b-4df2-9558-06712d66b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:55.902662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9e60f-4a7b-4df2-9558-06712d66b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.902949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9e60f-4a7b-4df2-9558-06712d66b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.903226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9e60f-4a7b-4df2-9558-06712d66b7b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.918488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9e60f-4a7b-4df2-9558-06712d66b7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:55.920188 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9e60f-4a7b-4df2-9558-06712d66b7b0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:55.932804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:55.934819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:55.935266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:55.935686 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:55.936591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:55.938418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-aca9e60f-4a7b-4df2-9558-06712d66b7b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:49:55.942343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:55.943346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:55.944321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:49:55.944321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:49:55.944321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:55.944321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:55.944321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 240/475] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:55.946438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.sql.core [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 18:49:55.946438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 18:49:55.946438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [parameters: {'id': 'aa40d32902094d81ba89f1202cec5f3a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 18:49:55.946438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80157) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 18:49:55.950752 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9e60f-4a7b-4df2-9558-06712d66b7b0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result.read() Jun 21 18:49:55.967702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return ref.to_dict() Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:49:55.968818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application yield resource Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise self.value Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application session.commit() Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 18:49:55.969823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.session.flush() Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application compat.raise_( Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 18:49:55.970998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:49:55.972030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result.read() Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:49:55.973115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [parameters: {'id': 'aa40d32902094d81ba89f1202cec5f3a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:55.975392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 18:49:55.978813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:55.981737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:55.981881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 236/476] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jun 21 18:49:55 2022] POST /identity/v3/roles => generated 144 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:55.984803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:56.096809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:56.104342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9e60f-4a7b-4df2-9558-06712d66b7b0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:56.144511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0745797-d50c-46f2-aad8-5861bc18d75b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:56.149888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75418a-745e-497e-84a4-fa3e3c178048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:56.150565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75418a-745e-497e-84a4-fa3e3c178048 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:56.150855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75418a-745e-497e-84a4-fa3e3c178048 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:56.151133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75418a-745e-497e-84a4-fa3e3c178048 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:56.152418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75418a-745e-497e-84a4-fa3e3c178048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:56.152726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75418a-745e-497e-84a4-fa3e3c178048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:56.155902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75418a-745e-497e-84a4-fa3e3c178048 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:56.170639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 241/477] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:49:56.176452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75418a-745e-497e-84a4-fa3e3c178048 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:56.200961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15e5d7-2c8e-4a85-a851-911f86fec261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:56.201812 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15e5d7-2c8e-4a85-a851-911f86fec261 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:56.202090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15e5d7-2c8e-4a85-a851-911f86fec261 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:56.202365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15e5d7-2c8e-4a85-a851-911f86fec261 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:56.226691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15e5d7-2c8e-4a85-a851-911f86fec261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3a15e5d7-2c8e-4a85-a851-911f86fec261 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:56.234749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:56.235662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:56.237309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 237/478] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:49:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:56.246010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15e5d7-2c8e-4a85-a851-911f86fec261 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:56.311248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:56.387962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75418a-745e-497e-84a4-fa3e3c178048 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:56.453615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15e5d7-2c8e-4a85-a851-911f86fec261 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:56.488509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:56.495134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b0da-6a94-4daa-870a-fbd6baaf9b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:56.496389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b0da-6a94-4daa-870a-fbd6baaf9b76 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:56.496930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b0da-6a94-4daa-870a-fbd6baaf9b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:56.497305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b0da-6a94-4daa-870a-fbd6baaf9b76 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:56.497862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc19f5c-27e5-45cc-ad84-f783fe4485f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:56.498324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4b0da-6a94-4daa-870a-fbd6baaf9b76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:56.498402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4b0da-6a94-4daa-870a-fbd6baaf9b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:56.498579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc19f5c-27e5-45cc-ad84-f783fe4485f2 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:56.498831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc19f5c-27e5-45cc-ad84-f783fe4485f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:56.499108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc19f5c-27e5-45cc-ad84-f783fe4485f2 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:56.499730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4b0da-6a94-4daa-870a-fbd6baaf9b76 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:56.499842 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc19f5c-27e5-45cc-ad84-f783fe4485f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:56.501271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc19f5c-27e5-45cc-ad84-f783fe4485f2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:56.509744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 238/479] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:49:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:56.540205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 242/480] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:49:56 2022] POST /identity/v3/users => generated 249 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:49:57.976646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c65011-f1ea-4a28-93ea-0ea9a5e95224 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:57.977191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c65011-f1ea-4a28-93ea-0ea9a5e95224 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:57.977269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c65011-f1ea-4a28-93ea-0ea9a5e95224 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:57.978905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 239/481] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:57.983989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22011ef-f098-4df5-88e7-922f86fffe47 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:57.984444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22011ef-f098-4df5-88e7-922f86fffe47 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:57.984617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22011ef-f098-4df5-88e7-922f86fffe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.006426 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d22011ef-f098-4df5-88e7-922f86fffe47 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:58.007660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d22011ef-f098-4df5-88e7-922f86fffe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:58.103665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 243/482] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:58.121360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bdd3a-734f-4aad-8883-ed1bed3ded71 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:58.121970 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bdd3a-734f-4aad-8883-ed1bed3ded71 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:58.122445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bdd3a-734f-4aad-8883-ed1bed3ded71 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.124317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 240/483] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:58.129874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9fdc2-4b76-49fa-b98e-0737b7cd3fc3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:58.130172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9fdc2-4b76-49fa-b98e-0737b7cd3fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:58.130459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9fdc2-4b76-49fa-b98e-0737b7cd3fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.135372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bdd3a-734f-4aad-8883-ed1bed3ded71 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:58.155447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c6a9fdc2-4b76-49fa-b98e-0737b7cd3fc3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:58.156423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c6a9fdc2-4b76-49fa-b98e-0737b7cd3fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:58.258936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bdd3a-734f-4aad-8883-ed1bed3ded71 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:58.293474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:58.298337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa837c-1fee-4d61-ab77-5b985ef1c0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:58.299398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa837c-1fee-4d61-ab77-5b985ef1c0cb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:58.299880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa837c-1fee-4d61-ab77-5b985ef1c0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:58.300493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa837c-1fee-4d61-ab77-5b985ef1c0cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.314226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa837c-1fee-4d61-ab77-5b985ef1c0cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-f6fa837c-1fee-4d61-ab77-5b985ef1c0cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:58.332926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:58.333847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:49:58.334713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:58.335334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 244/484] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:58.345503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 241/485] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:49:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:58.362004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9fdc2-4b76-49fa-b98e-0737b7cd3fc3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:58.366402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa837c-1fee-4d61-ab77-5b985ef1c0cb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:58.461603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:58.563536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa837c-1fee-4d61-ab77-5b985ef1c0cb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:58.567944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9fdc2-4b76-49fa-b98e-0737b7cd3fc3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:58.617200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6aaff-8eaa-4656-8ceb-093b743f662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:58.617885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6aaff-8eaa-4656-8ceb-093b743f662b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:58.618261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6aaff-8eaa-4656-8ceb-093b743f662b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:58.618551 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6aaff-8eaa-4656-8ceb-093b743f662b None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.619418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6aaff-8eaa-4656-8ceb-093b743f662b None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:58.619822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6aaff-8eaa-4656-8ceb-093b743f662b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:58.621482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a80bc-9d42-4730-8f46-468d000f033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:58.621605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6aaff-8eaa-4656-8ceb-093b743f662b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:58.622579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a80bc-9d42-4730-8f46-468d000f033d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:58.623094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a80bc-9d42-4730-8f46-468d000f033d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:58.623559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a80bc-9d42-4730-8f46-468d000f033d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.630942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 245/486] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:49:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:58.635666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a80bc-9d42-4730-8f46-468d000f033d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:58.637261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68597d-b367-4f92-b8db-09719820ab2b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:58.637557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68597d-b367-4f92-b8db-09719820ab2b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:58.637839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68597d-b367-4f92-b8db-09719820ab2b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.639444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 246/487] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:58.640150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68597d-b367-4f92-b8db-09719820ab2b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c23a80bc-9d42-4730-8f46-468d000f033d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:58.645425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:58.646321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:49:58.647123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:58.649840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 242/488] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:49:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:49:58.656393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dc981-c1a2-4e14-a96c-6b40095c1d84 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:58.657141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dc981-c1a2-4e14-a96c-6b40095c1d84 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:58.657630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dc981-c1a2-4e14-a96c-6b40095c1d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.676211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6c6dc981-c1a2-4e14-a96c-6b40095c1d84 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:58.677773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6c6dc981-c1a2-4e14-a96c-6b40095c1d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:58.701578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:58.752792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 243/489] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:58.753977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dc981-c1a2-4e14-a96c-6b40095c1d84 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:58.777257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68597d-b367-4f92-b8db-09719820ab2b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:58.819533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9383b8c-1840-4304-9610-34ecb19a4855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:58.820697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9383b8c-1840-4304-9610-34ecb19a4855 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:58.821295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9383b8c-1840-4304-9610-34ecb19a4855 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:58.821781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9383b8c-1840-4304-9610-34ecb19a4855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.829151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:58.836431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9383b8c-1840-4304-9610-34ecb19a4855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:58.838411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9383b8c-1840-4304-9610-34ecb19a4855 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b9383b8c-1840-4304-9610-34ecb19a4855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:49:58.858525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:58.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:58.860290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:49:58.860290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:49:58.860290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:58.860290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:58.862731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 247/490] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:58.863782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9383b8c-1840-4304-9610-34ecb19a4855 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:58.898405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dc981-c1a2-4e14-a96c-6b40095c1d84 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:58.941623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b59b81-0c8b-4059-8dc0-c001720ed45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:58.942352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b59b81-0c8b-4059-8dc0-c001720ed45b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:58.942644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b59b81-0c8b-4059-8dc0-c001720ed45b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:58.942926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b59b81-0c8b-4059-8dc0-c001720ed45b None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:58.943954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b59b81-0c8b-4059-8dc0-c001720ed45b None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:58.944267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b59b81-0c8b-4059-8dc0-c001720ed45b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:58.946489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b59b81-0c8b-4059-8dc0-c001720ed45b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:58.963539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 244/491] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:49:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:58.963539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b59b81-0c8b-4059-8dc0-c001720ed45b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:58.996917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:59.001227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9383b8c-1840-4304-9610-34ecb19a4855 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.038654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a2c6a-1349-42b5-808c-e3f4d45000b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.039465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a2c6a-1349-42b5-808c-e3f4d45000b7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.039748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a2c6a-1349-42b5-808c-e3f4d45000b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.040024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a2c6a-1349-42b5-808c-e3f4d45000b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.050676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a2c6a-1349-42b5-808c-e3f4d45000b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-690a2c6a-1349-42b5-808c-e3f4d45000b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:59.059034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:59.060107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:49:59.061112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:59.065323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 248/492] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:49:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:59.066206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a2c6a-1349-42b5-808c-e3f4d45000b7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.073164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b59b81-0c8b-4059-8dc0-c001720ed45b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.077469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:59.117337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d63be5-0484-481b-8414-73d65b385ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.118003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d63be5-0484-481b-8414-73d65b385ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.118290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d63be5-0484-481b-8414-73d65b385ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.118571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d63be5-0484-481b-8414-73d65b385ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.119299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d63be5-0484-481b-8414-73d65b385ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:59.119612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d63be5-0484-481b-8414-73d65b385ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:59.121557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d63be5-0484-481b-8414-73d65b385ab5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:59.133606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 245/493] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:59.134869 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34b991-2a78-4273-af0c-7c9eb3d99154 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.135166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34b991-2a78-4273-af0c-7c9eb3d99154 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.135447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34b991-2a78-4273-af0c-7c9eb3d99154 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.137436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 246/494] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:59.138012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34b991-2a78-4273-af0c-7c9eb3d99154 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.179743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a2c6a-1349-42b5-808c-e3f4d45000b7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.214576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ecfd4-f5e2-4af8-982a-0b0fe0bc6029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.215962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ecfd4-f5e2-4af8-982a-0b0fe0bc6029 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.216250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ecfd4-f5e2-4af8-982a-0b0fe0bc6029 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.216530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ecfd4-f5e2-4af8-982a-0b0fe0bc6029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.228558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ecfd4-f5e2-4af8-982a-0b0fe0bc6029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:59.230633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ecfd4-f5e2-4af8-982a-0b0fe0bc6029 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:59.242693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34b991-2a78-4273-af0c-7c9eb3d99154 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-513ecfd4-f5e2-4af8-982a-0b0fe0bc6029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:49:59.249001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:59.250129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:59.255590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:49:59.255590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:49:59.255590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:59.255590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:59.255590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 249/495] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:59.255590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce10a5-714d-4fa1-bcbe-82568d2c67ce None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.255590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce10a5-714d-4fa1-bcbe-82568d2c67ce None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.255590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce10a5-714d-4fa1-bcbe-82568d2c67ce None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.255590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 250/496] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:49:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:49:59.256172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce10a5-714d-4fa1-bcbe-82568d2c67ce None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.272049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:59.284844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b43c95-9250-4234-a605-b57c8828cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.285550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b43c95-9250-4234-a605-b57c8828cc5f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.285841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b43c95-9250-4234-a605-b57c8828cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.286121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b43c95-9250-4234-a605-b57c8828cc5f None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.286878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b43c95-9250-4234-a605-b57c8828cc5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:59.287194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b43c95-9250-4234-a605-b57c8828cc5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:59.288576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b43c95-9250-4234-a605-b57c8828cc5f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:59.297288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 247/497] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:59.301798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91739c6-c83c-4a1f-a92e-c8922b44c1bf None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.301798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91739c6-c83c-4a1f-a92e-c8922b44c1bf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.301798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91739c6-c83c-4a1f-a92e-c8922b44c1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.316019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a91739c6-c83c-4a1f-a92e-c8922b44c1bf None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:59.317242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a91739c6-c83c-4a1f-a92e-c8922b44c1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:59.339453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:59.364077 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce10a5-714d-4fa1-bcbe-82568d2c67ce None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.387257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 248/498] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:59.394719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91739c6-c83c-4a1f-a92e-c8922b44c1bf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.405245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9234c-3f46-4b16-91c2-1eae22c833eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.406047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9234c-3f46-4b16-91c2-1eae22c833eb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.406491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9234c-3f46-4b16-91c2-1eae22c833eb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.406772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9234c-3f46-4b16-91c2-1eae22c833eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.416092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9234c-3f46-4b16-91c2-1eae22c833eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:59.418135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9234c-3f46-4b16-91c2-1eae22c833eb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-cdd9234c-3f46-4b16-91c2-1eae22c833eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:49:59.434360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:59.435383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:59.436287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 251/499] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:59.436873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e774c7-7224-4811-83f6-497a14755827 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.437068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e774c7-7224-4811-83f6-497a14755827 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.438624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e774c7-7224-4811-83f6-497a14755827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.455393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-67e774c7-7224-4811-83f6-497a14755827 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:49:59.456586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-67e774c7-7224-4811-83f6-497a14755827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:49:59.496291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91739c6-c83c-4a1f-a92e-c8922b44c1bf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.523036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 252/500] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:49:59.523887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e774c7-7224-4811-83f6-497a14755827 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.526127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:59.533139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c7f97-7937-4073-ab94-851d85e87ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.534011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c7f97-7937-4073-ab94-851d85e87ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.534121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c7f97-7937-4073-ab94-851d85e87ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.534348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c7f97-7937-4073-ab94-851d85e87ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.535075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c7f97-7937-4073-ab94-851d85e87ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:59.535383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c7f97-7937-4073-ab94-851d85e87ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:59.536770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c7f97-7937-4073-ab94-851d85e87ca8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:59.548576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 249/501] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:59.549510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c7f97-7937-4073-ab94-851d85e87ca8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.568186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:59.626751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e774c7-7224-4811-83f6-497a14755827 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.650870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c7f97-7937-4073-ab94-851d85e87ca8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.663158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719dd17-3133-4464-9248-25b3a4d21c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.663840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719dd17-3133-4464-9248-25b3a4d21c49 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.664247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719dd17-3133-4464-9248-25b3a4d21c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.664530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719dd17-3133-4464-9248-25b3a4d21c49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.676352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719dd17-3133-4464-9248-25b3a4d21c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:59.677869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719dd17-3133-4464-9248-25b3a4d21c49 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:59.689844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91f95f-e13a-4f2a-aa9e-5b43d2053e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.690699 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91f95f-e13a-4f2a-aa9e-5b43d2053e64 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.690926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91f95f-e13a-4f2a-aa9e-5b43d2053e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.691334 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91f95f-e13a-4f2a-aa9e-5b43d2053e64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-4719dd17-3133-4464-9248-25b3a4d21c49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:49:59.699348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:59.700195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:59.701063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:49:59.701063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:49:59.701063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:59.701063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:49:59.701063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 253/502] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:59.701380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719dd17-3133-4464-9248-25b3a4d21c49 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.707764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91f95f-e13a-4f2a-aa9e-5b43d2053e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:59.709273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91f95f-e13a-4f2a-aa9e-5b43d2053e64 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6a91f95f-e13a-4f2a-aa9e-5b43d2053e64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:49:59.726372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:59.727238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:59.728081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:49:59.728081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:49:59.728081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:49:59.728081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:59.728081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 250/503] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:59.728379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91f95f-e13a-4f2a-aa9e-5b43d2053e64 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.760788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:59.794929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:49:59.805718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719dd17-3133-4464-9248-25b3a4d21c49 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.830850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91f95f-e13a-4f2a-aa9e-5b43d2053e64 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.840639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dcfbe-89b1-49c9-a02d-e3f20fe43079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.841340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dcfbe-89b1-49c9-a02d-e3f20fe43079 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.841626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dcfbe-89b1-49c9-a02d-e3f20fe43079 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.841907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dcfbe-89b1-49c9-a02d-e3f20fe43079 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.842744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dcfbe-89b1-49c9-a02d-e3f20fe43079 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:49:59.843067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dcfbe-89b1-49c9-a02d-e3f20fe43079 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:49:59.844526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dcfbe-89b1-49c9-a02d-e3f20fe43079 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:49:59.864533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4c4b5-1ae2-4e4f-9ca9-b8207fff5230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:49:59.865357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4c4b5-1ae2-4e4f-9ca9-b8207fff5230 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:49:59.865645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4c4b5-1ae2-4e4f-9ca9-b8207fff5230 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:49:59.865921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4c4b5-1ae2-4e4f-9ca9-b8207fff5230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:49:59.876707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4c4b5-1ae2-4e4f-9ca9-b8207fff5230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:49:59.881803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 254/504] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:49:59.882590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dcfbe-89b1-49c9-a02d-e3f20fe43079 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-92e4c4b5-1ae2-4e4f-9ca9-b8207fff5230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:49:59.884436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:49:59.885315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:49:59.886703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 251/505] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:49:59.887352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4c4b5-1ae2-4e4f-9ca9-b8207fff5230 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:49:59.986959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dcfbe-89b1-49c9-a02d-e3f20fe43079 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:49:59.991238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4c4b5-1ae2-4e4f-9ca9-b8207fff5230 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.000669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:00.017913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:00.022545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bd1fd-c320-4e9b-b815-f9795c5eee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.023346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bd1fd-c320-4e9b-b815-f9795c5eee55 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.023632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bd1fd-c320-4e9b-b815-f9795c5eee55 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.023912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bd1fd-c320-4e9b-b815-f9795c5eee55 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.026437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd728bf-7473-4f5f-b69e-855723cfad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.027112 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd728bf-7473-4f5f-b69e-855723cfad70 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.027394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd728bf-7473-4f5f-b69e-855723cfad70 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.027670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd728bf-7473-4f5f-b69e-855723cfad70 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.028386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd728bf-7473-4f5f-b69e-855723cfad70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:00.028693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd728bf-7473-4f5f-b69e-855723cfad70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.030092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd728bf-7473-4f5f-b69e-855723cfad70 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.031315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bd1fd-c320-4e9b-b815-f9795c5eee55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.032747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bd1fd-c320-4e9b-b815-f9795c5eee55 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.042752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 252/506] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:00.043631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd728bf-7473-4f5f-b69e-855723cfad70 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-8f1bd1fd-c320-4e9b-b815-f9795c5eee55 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:00.048105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:00.049143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:00.050022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 255/507] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:49:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:00.050574 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bd1fd-c320-4e9b-b815-f9795c5eee55 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.143816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd728bf-7473-4f5f-b69e-855723cfad70 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.153823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bd1fd-c320-4e9b-b815-f9795c5eee55 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.177661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c81d2-396f-4cc5-8c46-e96431f50a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.178316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c81d2-396f-4cc5-8c46-e96431f50a32 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.178598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c81d2-396f-4cc5-8c46-e96431f50a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.178873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c81d2-396f-4cc5-8c46-e96431f50a32 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.179595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c81d2-396f-4cc5-8c46-e96431f50a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:00.180141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c81d2-396f-4cc5-8c46-e96431f50a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.181347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c81d2-396f-4cc5-8c46-e96431f50a32 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.188158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86036675-a8d2-4042-86ff-7009e4436d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.188813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86036675-a8d2-4042-86ff-7009e4436d6c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.189108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86036675-a8d2-4042-86ff-7009e4436d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.189396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86036675-a8d2-4042-86ff-7009e4436d6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.194004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 253/508] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:00.194822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c81d2-396f-4cc5-8c46-e96431f50a32 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.231236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:00.244038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:00.257978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86036675-a8d2-4042-86ff-7009e4436d6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.259437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86036675-a8d2-4042-86ff-7009e4436d6c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.295576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c81d2-396f-4cc5-8c46-e96431f50a32 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.332703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a27dd6-60cd-43c5-a5b7-84e1e5f6468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.333393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a27dd6-60cd-43c5-a5b7-84e1e5f6468a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.333678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a27dd6-60cd-43c5-a5b7-84e1e5f6468a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.333955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a27dd6-60cd-43c5-a5b7-84e1e5f6468a None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.334703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a27dd6-60cd-43c5-a5b7-84e1e5f6468a None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:00.335011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a27dd6-60cd-43c5-a5b7-84e1e5f6468a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.336398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a27dd6-60cd-43c5-a5b7-84e1e5f6468a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.344967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 254/509] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:00.345630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a27dd6-60cd-43c5-a5b7-84e1e5f6468a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.365397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 256/510] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:00.366220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86036675-a8d2-4042-86ff-7009e4436d6c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.436195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:00.443704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a27dd6-60cd-43c5-a5b7-84e1e5f6468a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.454412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:00.466925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86036675-a8d2-4042-86ff-7009e4436d6c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.479064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755641c-afdc-45b4-bd84-eee6d0dea17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.479877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755641c-afdc-45b4-bd84-eee6d0dea17f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.480185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755641c-afdc-45b4-bd84-eee6d0dea17f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.480460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755641c-afdc-45b4-bd84-eee6d0dea17f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.487901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755641c-afdc-45b4-bd84-eee6d0dea17f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.489374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755641c-afdc-45b4-bd84-eee6d0dea17f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.499793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5c572-f9f5-4651-a607-225a30bf3b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.500450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5c572-f9f5-4651-a607-225a30bf3b1d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.500730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5c572-f9f5-4651-a607-225a30bf3b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.501004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5c572-f9f5-4651-a607-225a30bf3b1d None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.501854 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5c572-f9f5-4651-a607-225a30bf3b1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:00.502160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5c572-f9f5-4651-a607-225a30bf3b1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.503573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5c572-f9f5-4651-a607-225a30bf3b1d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c755641c-afdc-45b4-bd84-eee6d0dea17f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:00.504807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:00.505859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 255/511] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:00.506961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755641c-afdc-45b4-bd84-eee6d0dea17f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.524686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 257/512] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:00.525531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5c572-f9f5-4651-a607-225a30bf3b1d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.607077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755641c-afdc-45b4-bd84-eee6d0dea17f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.622179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5c572-f9f5-4651-a607-225a30bf3b1d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.640691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07670502-10b1-4cc5-8912-560988c458f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.641383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07670502-10b1-4cc5-8912-560988c458f1 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.641672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07670502-10b1-4cc5-8912-560988c458f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.641952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07670502-10b1-4cc5-8912-560988c458f1 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.642659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07670502-10b1-4cc5-8912-560988c458f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.644119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07670502-10b1-4cc5-8912-560988c458f1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.650700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:00.657605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb216dd-3208-4deb-81e7-159553d540aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.658417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb216dd-3208-4deb-81e7-159553d540aa None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.658767 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb216dd-3208-4deb-81e7-159553d540aa None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.658999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb216dd-3208-4deb-81e7-159553d540aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.663292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 256/513] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:50:00 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:00.673532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07670502-10b1-4cc5-8912-560988c458f1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.673665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb216dd-3208-4deb-81e7-159553d540aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.673665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb216dd-3208-4deb-81e7-159553d540aa None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-9fb216dd-3208-4deb-81e7-159553d540aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:00.695788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:00.696884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:00.697955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:00.697955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:00.697955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:00.697955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:00.698708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 258/514] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:00.699387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb216dd-3208-4deb-81e7-159553d540aa None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.713067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:00.805413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07670502-10b1-4cc5-8912-560988c458f1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.817171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb216dd-3208-4deb-81e7-159553d540aa None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:00.849688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5e4d6-5f47-4fc9-bcf1-1a9bcdbfc4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.850495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5e4d6-5f47-4fc9-bcf1-1a9bcdbfc4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.850776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5e4d6-5f47-4fc9-bcf1-1a9bcdbfc4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.851052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5e4d6-5f47-4fc9-bcf1-1a9bcdbfc4c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.861365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bd902-6fd0-4dce-ae8a-c1a626bcf3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:00.862857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bd902-6fd0-4dce-ae8a-c1a626bcf3bc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:00.863287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bd902-6fd0-4dce-ae8a-c1a626bcf3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:00.863712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bd902-6fd0-4dce-ae8a-c1a626bcf3bc None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:00.865023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bd902-6fd0-4dce-ae8a-c1a626bcf3bc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:00.865412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5e4d6-5f47-4fc9-bcf1-1a9bcdbfc4c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.865705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bd902-6fd0-4dce-ae8a-c1a626bcf3bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:00.867095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5e4d6-5f47-4fc9-bcf1-1a9bcdbfc4c1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.867468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bd902-6fd0-4dce-ae8a-c1a626bcf3bc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-74d5e4d6-5f47-4fc9-bcf1-1a9bcdbfc4c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:00.886676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:00.887768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:00.888805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:00.888805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:00.888805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:00.888805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:00.893049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 257/515] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:00.893049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5e4d6-5f47-4fc9-bcf1-1a9bcdbfc4c1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.893294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 259/516] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:00.893294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bd902-6fd0-4dce-ae8a-c1a626bcf3bc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:00.943483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:00.982618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:01.006882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bd902-6fd0-4dce-ae8a-c1a626bcf3bc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.009364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5e4d6-5f47-4fc9-bcf1-1a9bcdbfc4c1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.045836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec290f9-4628-45ee-a2c3-02649644af39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.046833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181ffb-7b46-4518-bda8-53bc9592f2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.047112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec290f9-4628-45ee-a2c3-02649644af39 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.047480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181ffb-7b46-4518-bda8-53bc9592f2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.047771 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181ffb-7b46-4518-bda8-53bc9592f2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.047873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec290f9-4628-45ee-a2c3-02649644af39 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.048047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181ffb-7b46-4518-bda8-53bc9592f2a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.048486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec290f9-4628-45ee-a2c3-02649644af39 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.049416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec290f9-4628-45ee-a2c3-02649644af39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:01.049903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec290f9-4628-45ee-a2c3-02649644af39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.051473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec290f9-4628-45ee-a2c3-02649644af39 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.068980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 260/517] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:01.068980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec290f9-4628-45ee-a2c3-02649644af39 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:01.130232 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85181ffb-7b46-4518-bda8-53bc9592f2a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.132194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85181ffb-7b46-4518-bda8-53bc9592f2a3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.176746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:01.179944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec290f9-4628-45ee-a2c3-02649644af39 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.213822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72f6d4-c164-4d88-ad7d-76dd96d50f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.215020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72f6d4-c164-4d88-ad7d-76dd96d50f66 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.215308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72f6d4-c164-4d88-ad7d-76dd96d50f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.215582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72f6d4-c164-4d88-ad7d-76dd96d50f66 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.216310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b72f6d4-c164-4d88-ad7d-76dd96d50f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:01.216621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b72f6d4-c164-4d88-ad7d-76dd96d50f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.218043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b72f6d4-c164-4d88-ad7d-76dd96d50f66 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.221610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:01.235179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 261/518] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:01.235179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72f6d4-c164-4d88-ad7d-76dd96d50f66 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:01.257129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 258/519] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:01.257129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181ffb-7b46-4518-bda8-53bc9592f2a3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:01.347306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72f6d4-c164-4d88-ad7d-76dd96d50f66 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.366174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181ffb-7b46-4518-bda8-53bc9592f2a3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.384647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8c29b-d262-4fb6-afee-5872abfa4dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.385336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c29b-d262-4fb6-afee-5872abfa4dfd None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.385623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c29b-d262-4fb6-afee-5872abfa4dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.385940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c29b-d262-4fb6-afee-5872abfa4dfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.401270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26706e2-7405-4a45-9e02-4e01adf8e5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.402617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26706e2-7405-4a45-9e02-4e01adf8e5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.402904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26706e2-7405-4a45-9e02-4e01adf8e5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.403183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26706e2-7405-4a45-9e02-4e01adf8e5f3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.410653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26706e2-7405-4a45-9e02-4e01adf8e5f3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.412278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26706e2-7405-4a45-9e02-4e01adf8e5f3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.418456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c26706e2-7405-4a45-9e02-4e01adf8e5f3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:01.428124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:01.429032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:50:01.430090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:01.433902 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 259/520] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:01.434733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26706e2-7405-4a45-9e02-4e01adf8e5f3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:01.468690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8c29b-d262-4fb6-afee-5872abfa4dfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.470464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8c29b-d262-4fb6-afee-5872abfa4dfd None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.477853 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:01.541832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26706e2-7405-4a45-9e02-4e01adf8e5f3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.578731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41daaa4-e74e-4cee-be1a-b9742874c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.579532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41daaa4-e74e-4cee-be1a-b9742874c32a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.579821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41daaa4-e74e-4cee-be1a-b9742874c32a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.580102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41daaa4-e74e-4cee-be1a-b9742874c32a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.582275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 262/521] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:01.582831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8c29b-d262-4fb6-afee-5872abfa4dfd None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:01.592196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41daaa4-e74e-4cee-be1a-b9742874c32a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.594203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41daaa4-e74e-4cee-be1a-b9742874c32a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-f41daaa4-e74e-4cee-be1a-b9742874c32a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:01.610225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:01.611577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 260/522] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:01.612815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41daaa4-e74e-4cee-be1a-b9742874c32a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:01.650424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:01.697696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8c29b-d262-4fb6-afee-5872abfa4dfd None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.715371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:01.719549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41daaa4-e74e-4cee-be1a-b9742874c32a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.736294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7524b7f-3566-4494-847f-80f002d4adb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.737235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524b7f-3566-4494-847f-80f002d4adb5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.737643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524b7f-3566-4494-847f-80f002d4adb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.737932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524b7f-3566-4494-847f-80f002d4adb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.750172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7524b7f-3566-4494-847f-80f002d4adb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.751877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7524b7f-3566-4494-847f-80f002d4adb5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.756157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8308763-c8de-4ec0-85f6-a5617be72b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.756836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8308763-c8de-4ec0-85f6-a5617be72b90 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.757146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8308763-c8de-4ec0-85f6-a5617be72b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.757431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8308763-c8de-4ec0-85f6-a5617be72b90 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.758283 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8308763-c8de-4ec0-85f6-a5617be72b90 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:01.758596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8308763-c8de-4ec0-85f6-a5617be72b90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.760033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8308763-c8de-4ec0-85f6-a5617be72b90 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b7524b7f-3566-4494-847f-80f002d4adb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:01.771819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:01.772883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:01.774085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:01.774085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:01.774085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:01.774085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:01.774085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 263/523] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:01.774085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7524b7f-3566-4494-847f-80f002d4adb5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:01.803678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 261/524] 10.222.0.70 () {56 vars in 1139 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:01.804875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8308763-c8de-4ec0-85f6-a5617be72b90 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:01.893199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7524b7f-3566-4494-847f-80f002d4adb5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.907665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:01.917689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8308763-c8de-4ec0-85f6-a5617be72b90 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:01.932435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a9ae0-e1e5-4077-9056-42ea48cca292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.934248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a9ae0-e1e5-4077-9056-42ea48cca292 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.934716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a9ae0-e1e5-4077-9056-42ea48cca292 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.935170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a9ae0-e1e5-4077-9056-42ea48cca292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.946835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a9ae0-e1e5-4077-9056-42ea48cca292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.948544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a9ae0-e1e5-4077-9056-42ea48cca292 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.954230 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db98088-d280-438f-94a5-3bb9117276cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:01.954895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db98088-d280-438f-94a5-3bb9117276cd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:01.955198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db98088-d280-438f-94a5-3bb9117276cd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:01.955472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db98088-d280-438f-94a5-3bb9117276cd None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:01.956271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db98088-d280-438f-94a5-3bb9117276cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:01.956584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db98088-d280-438f-94a5-3bb9117276cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:01.958065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db98088-d280-438f-94a5-3bb9117276cd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a12a9ae0-e1e5-4077-9056-42ea48cca292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:01.970305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:01.971185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:01.972116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:01.972116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:01.972116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:01.972116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:01.974122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 264/525] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:01.974930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a9ae0-e1e5-4077-9056-42ea48cca292 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:01.981162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 262/526] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:01.982204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db98088-d280-438f-94a5-3bb9117276cd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:02.005009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:02.082649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a9ae0-e1e5-4077-9056-42ea48cca292 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.094742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db98088-d280-438f-94a5-3bb9117276cd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.119940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de964da9-e0f9-4db2-b40e-d61f66c97996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:02.120638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de964da9-e0f9-4db2-b40e-d61f66c97996 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.120923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de964da9-e0f9-4db2-b40e-d61f66c97996 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.121231 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de964da9-e0f9-4db2-b40e-d61f66c97996 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.121954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de964da9-e0f9-4db2-b40e-d61f66c97996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:02.122267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de964da9-e0f9-4db2-b40e-d61f66c97996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:02.123630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de964da9-e0f9-4db2-b40e-d61f66c97996 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:02.131616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eeac9-16a1-47ba-9cc1-87fa1ff0caa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:02.132268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eeac9-16a1-47ba-9cc1-87fa1ff0caa4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.132649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eeac9-16a1-47ba-9cc1-87fa1ff0caa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.132936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eeac9-16a1-47ba-9cc1-87fa1ff0caa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.135902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 265/527] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:02.136913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de964da9-e0f9-4db2-b40e-d61f66c97996 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:02.165524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:02.212523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eeac9-16a1-47ba-9cc1-87fa1ff0caa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:02.214880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eeac9-16a1-47ba-9cc1-87fa1ff0caa4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:02.233568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:02.244847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de964da9-e0f9-4db2-b40e-d61f66c97996 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.280090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19d826-ddc9-42ee-adab-c7a9154198f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:02.280769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19d826-ddc9-42ee-adab-c7a9154198f7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.281184 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19d826-ddc9-42ee-adab-c7a9154198f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.281477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19d826-ddc9-42ee-adab-c7a9154198f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.282205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19d826-ddc9-42ee-adab-c7a9154198f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:02.282518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19d826-ddc9-42ee-adab-c7a9154198f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:02.283894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19d826-ddc9-42ee-adab-c7a9154198f7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:02.297308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 266/528] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:02.297992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19d826-ddc9-42ee-adab-c7a9154198f7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:02.338620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 263/529] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:02.344157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eeac9-16a1-47ba-9cc1-87fa1ff0caa4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:02.391556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:02.409362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19d826-ddc9-42ee-adab-c7a9154198f7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.447086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f052f4-c00c-4dd2-b038-63fb868b8997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:02.447744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f052f4-c00c-4dd2-b038-63fb868b8997 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.448034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f052f4-c00c-4dd2-b038-63fb868b8997 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.448859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f052f4-c00c-4dd2-b038-63fb868b8997 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.449731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f052f4-c00c-4dd2-b038-63fb868b8997 None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=dcbcec65220c4ebb95f974925fa73f34) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:02.450049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f052f4-c00c-4dd2-b038-63fb868b8997 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:02.451486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f052f4-c00c-4dd2-b038-63fb868b8997 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:02.454938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eeac9-16a1-47ba-9cc1-87fa1ff0caa4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.456926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 267/530] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:02 2022] GET /identity/v3/role_assignments?user.id=dcbcec65220c4ebb95f974925fa73f34&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:02.457738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f052f4-c00c-4dd2-b038-63fb868b8997 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:02.483272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:02.513156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21332e03-73ae-4947-912b-10c049e8c78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:02.514995 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21332e03-73ae-4947-912b-10c049e8c78d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.515483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21332e03-73ae-4947-912b-10c049e8c78d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.516478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21332e03-73ae-4947-912b-10c049e8c78d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.528213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21332e03-73ae-4947-912b-10c049e8c78d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:02.530687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21332e03-73ae-4947-912b-10c049e8c78d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-21332e03-73ae-4947-912b-10c049e8c78d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:02.552146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:02.553285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:02.554194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:02.554194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:02.554194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:02.554194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:02.554194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:02.554194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:02.554194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:02.554194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:02.557152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 264/531] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:02.559335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21332e03-73ae-4947-912b-10c049e8c78d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:02.586283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f052f4-c00c-4dd2-b038-63fb868b8997 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.629807 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4602d82-d11b-476a-9d99-d805d99b5b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:02.630874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4602d82-d11b-476a-9d99-d805d99b5b56 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.630874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4602d82-d11b-476a-9d99-d805d99b5b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.631099 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4602d82-d11b-476a-9d99-d805d99b5b56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.639815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4602d82-d11b-476a-9d99-d805d99b5b56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:02.641335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4602d82-d11b-476a-9d99-d805d99b5b56 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a4602d82-d11b-476a-9d99-d805d99b5b56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:02.662851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:02.663787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:02.664607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:02.664607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:02.664607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:02.664607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:02.664607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:02.664607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:02.664607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:02.664607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:02.669194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 268/532] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:02.670124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4602d82-d11b-476a-9d99-d805d99b5b56 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:02.686278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:02.703424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21332e03-73ae-4947-912b-10c049e8c78d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.746933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68c34-35ba-4eee-b710-961b3cc24918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:02.747728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68c34-35ba-4eee-b710-961b3cc24918 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.748029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68c34-35ba-4eee-b710-961b3cc24918 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.748292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68c34-35ba-4eee-b710-961b3cc24918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.760195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68c34-35ba-4eee-b710-961b3cc24918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:02.762836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68c34-35ba-4eee-b710-961b3cc24918 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2fd68c34-35ba-4eee-b710-961b3cc24918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:02.782051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:02.782971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:02.783810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:02.783810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:02.783810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:02.783810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:02.783810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 265/533] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:02.784080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68c34-35ba-4eee-b710-961b3cc24918 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:02.796343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4602d82-d11b-476a-9d99-d805d99b5b56 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.826873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:02.833445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b3ab8-8fd1-4ec5-b6d8-60facb49fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:02.834345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b3ab8-8fd1-4ec5-b6d8-60facb49fdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.834630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b3ab8-8fd1-4ec5-b6d8-60facb49fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.834912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b3ab8-8fd1-4ec5-b6d8-60facb49fdb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.835712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b3ab8-8fd1-4ec5-b6d8-60facb49fdb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:02.836028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b3ab8-8fd1-4ec5-b6d8-60facb49fdb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:02.837524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b3ab8-8fd1-4ec5-b6d8-60facb49fdb3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:02.856694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 269/534] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:02.858010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e59d1-08cf-4141-bd1a-837ca4a138d8 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.858303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e59d1-08cf-4141-bd1a-837ca4a138d8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.858582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e59d1-08cf-4141-bd1a-837ca4a138d8 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.859926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 270/535] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:02.864575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e59d1-08cf-4141-bd1a-837ca4a138d8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:02.913874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68c34-35ba-4eee-b710-961b3cc24918 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.939821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:02.957244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d27a5-06eb-4859-8013-fe11f027f3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:02.958227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d27a5-06eb-4859-8013-fe11f027f3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:02.958532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d27a5-06eb-4859-8013-fe11f027f3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:02.958827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d27a5-06eb-4859-8013-fe11f027f3d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:02.960074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d27a5-06eb-4859-8013-fe11f027f3d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:02.960413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d27a5-06eb-4859-8013-fe11f027f3d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:02.961978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d27a5-06eb-4859-8013-fe11f027f3d2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:02.975923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e59d1-08cf-4141-bd1a-837ca4a138d8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:02.982996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 266/536] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:02.982996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d27a5-06eb-4859-8013-fe11f027f3d2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:03.012731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac4cc5-9715-43a3-b969-22cb02eb5eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:03.013427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac4cc5-9715-43a3-b969-22cb02eb5eaf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:03.013708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac4cc5-9715-43a3-b969-22cb02eb5eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:03.013983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac4cc5-9715-43a3-b969-22cb02eb5eaf None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:03.014703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac4cc5-9715-43a3-b969-22cb02eb5eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:03.015008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac4cc5-9715-43a3-b969-22cb02eb5eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:03.016374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac4cc5-9715-43a3-b969-22cb02eb5eaf None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:03.028581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 271/537] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:03.029945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaaf9b-46eb-4ca6-9f76-ccb5d2c04e18 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:03.030234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaaf9b-46eb-4ca6-9f76-ccb5d2c04e18 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:03.030513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaaf9b-46eb-4ca6-9f76-ccb5d2c04e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:03.048353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-daaaaf9b-46eb-4ca6-9f76-ccb5d2c04e18 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:03.049565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-daaaaf9b-46eb-4ca6-9f76-ccb5d2c04e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:03.089846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:03.104534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d27a5-06eb-4859-8013-fe11f027f3d2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:03.124776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 272/538] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:03.124776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaaf9b-46eb-4ca6-9f76-ccb5d2c04e18 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:03.151207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e149ef-93ef-4600-a00a-358bc03b7434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:03.152854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e149ef-93ef-4600-a00a-358bc03b7434 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:03.153409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e149ef-93ef-4600-a00a-358bc03b7434 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:03.153886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e149ef-93ef-4600-a00a-358bc03b7434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:03.154863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e149ef-93ef-4600-a00a-358bc03b7434 None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=915704f26c814eaa9e6c2a68533a9c66) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:03.155344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e149ef-93ef-4600-a00a-358bc03b7434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:03.157008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e149ef-93ef-4600-a00a-358bc03b7434 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:03.163526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 267/539] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:02 2022] GET /identity/v3/role_assignments?user.id=915704f26c814eaa9e6c2a68533a9c66&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:03.164667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e149ef-93ef-4600-a00a-358bc03b7434 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:03.224159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:03.280833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaaf9b-46eb-4ca6-9f76-ccb5d2c04e18 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:03.297378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e149ef-93ef-4600-a00a-358bc03b7434 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:03.330413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ead04-2025-4347-98d6-5a652b843f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:03.331078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ead04-2025-4347-98d6-5a652b843f72 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:03.331360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ead04-2025-4347-98d6-5a652b843f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:03.331699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ead04-2025-4347-98d6-5a652b843f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:03.332615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ead04-2025-4347-98d6-5a652b843f72 None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=5b8161446e1344108f8283d46da23ab9) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:03.332893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ead04-2025-4347-98d6-5a652b843f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:03.334450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ead04-2025-4347-98d6-5a652b843f72 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:03.341047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af58301-c700-41bd-b0ce-720e9696495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:03.341410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 273/540] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:03 2022] GET /identity/v3/role_assignments?user.id=5b8161446e1344108f8283d46da23ab9&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:03.342234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ead04-2025-4347-98d6-5a652b843f72 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:03.345961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af58301-c700-41bd-b0ce-720e9696495a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:03.346294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af58301-c700-41bd-b0ce-720e9696495a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:03.346524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af58301-c700-41bd-b0ce-720e9696495a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:03.360391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af58301-c700-41bd-b0ce-720e9696495a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:03.370907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af58301-c700-41bd-b0ce-720e9696495a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-1af58301-c700-41bd-b0ce-720e9696495a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:03.409628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:03.410555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:03.411427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:03.411427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:03.411427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:03.411427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:03.411427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:03.411427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:03.411427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:03.411427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:03.411427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 268/541] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:03.422253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af58301-c700-41bd-b0ce-720e9696495a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:03.460032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ead04-2025-4347-98d6-5a652b843f72 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:03.472024 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:03.489443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:03.502300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9e0fe-dc2c-4547-b632-653fa395aa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:03.503113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9e0fe-dc2c-4547-b632-653fa395aa3c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:03.503399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9e0fe-dc2c-4547-b632-653fa395aa3c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:03.503680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9e0fe-dc2c-4547-b632-653fa395aa3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:03.518167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9e0fe-dc2c-4547-b632-653fa395aa3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-94a9e0fe-dc2c-4547-b632-653fa395aa3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:03.531060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:03.532036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:03.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:03.536864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 274/542] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:03.542184 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9e0fe-dc2c-4547-b632-653fa395aa3c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:03.678993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9e0fe-dc2c-4547-b632-653fa395aa3c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:03.695872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af58301-c700-41bd-b0ce-720e9696495a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:03.720190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bcada-48db-40e5-aed5-295f684b0e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:03.720859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bcada-48db-40e5-aed5-295f684b0e2c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:03.721167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bcada-48db-40e5-aed5-295f684b0e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:03.721447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bcada-48db-40e5-aed5-295f684b0e2c None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:03.722500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bcada-48db-40e5-aed5-295f684b0e2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:03.722809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bcada-48db-40e5-aed5-295f684b0e2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:03.724454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bcada-48db-40e5-aed5-295f684b0e2c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:03.734278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 275/543] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:03.742071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bcada-48db-40e5-aed5-295f684b0e2c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:03.768081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4311b2-4e9f-43d3-87bd-0f481dd60c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:03.768757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4311b2-4e9f-43d3-87bd-0f481dd60c6e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:03.769082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4311b2-4e9f-43d3-87bd-0f481dd60c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:03.769418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4311b2-4e9f-43d3-87bd-0f481dd60c6e None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:03.770253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4311b2-4e9f-43d3-87bd-0f481dd60c6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:03.770568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4311b2-4e9f-43d3-87bd-0f481dd60c6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:03.772007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4311b2-4e9f-43d3-87bd-0f481dd60c6e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:03.795531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 269/544] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:03.799751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4311b2-4e9f-43d3-87bd-0f481dd60c6e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:03.829511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:03.901057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:03.933447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bcada-48db-40e5-aed5-295f684b0e2c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:03.956931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4311b2-4e9f-43d3-87bd-0f481dd60c6e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:03.998928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91dfeb-c79c-4371-8fae-0c1fdb08cdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:03.999829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91dfeb-c79c-4371-8fae-0c1fdb08cdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:04.000116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91dfeb-c79c-4371-8fae-0c1fdb08cdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:04.000390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91dfeb-c79c-4371-8fae-0c1fdb08cdd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:04.007553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1111a4-88b5-468d-bf9d-4f490858109b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:04.008551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1111a4-88b5-468d-bf9d-4f490858109b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:04.008840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1111a4-88b5-468d-bf9d-4f490858109b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:04.009137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1111a4-88b5-468d-bf9d-4f490858109b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:04.009985 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1111a4-88b5-468d-bf9d-4f490858109b None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=808b530d24b3487b9a20ee74c131502e) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:04.010298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1111a4-88b5-468d-bf9d-4f490858109b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:04.012312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1111a4-88b5-468d-bf9d-4f490858109b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:04.019302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 270/545] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:03 2022] GET /identity/v3/role_assignments?user.id=808b530d24b3487b9a20ee74c131502e&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:04.028267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91dfeb-c79c-4371-8fae-0c1fdb08cdd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:04.029799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91dfeb-c79c-4371-8fae-0c1fdb08cdd7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-2b91dfeb-c79c-4371-8fae-0c1fdb08cdd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:04.057153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:04.058812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:04.059866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:04.059866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:04.059866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:04.059866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:04.059866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 276/546] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:04.066385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1111a4-88b5-468d-bf9d-4f490858109b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:04.201058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1111a4-88b5-468d-bf9d-4f490858109b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:04.242434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81444bb7-4373-482f-959a-a98396b7fe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:04.243360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81444bb7-4373-482f-959a-a98396b7fe0c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:04.243798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81444bb7-4373-482f-959a-a98396b7fe0c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:04.244250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81444bb7-4373-482f-959a-a98396b7fe0c None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:04.245229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81444bb7-4373-482f-959a-a98396b7fe0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:04.245691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81444bb7-4373-482f-959a-a98396b7fe0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:04.247241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81444bb7-4373-482f-959a-a98396b7fe0c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:04.259680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:04.268290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 271/547] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:04.274056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91dfeb-c79c-4371-8fae-0c1fdb08cdd7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:04.424304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:04.431324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91dfeb-c79c-4371-8fae-0c1fdb08cdd7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:04.487593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38994638-af22-4f13-94ee-505ee83fb2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:04.488330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38994638-af22-4f13-94ee-505ee83fb2cb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:04.488617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38994638-af22-4f13-94ee-505ee83fb2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:04.488895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38994638-af22-4f13-94ee-505ee83fb2cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:04.503690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38994638-af22-4f13-94ee-505ee83fb2cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:04.505184 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38994638-af22-4f13-94ee-505ee83fb2cb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-38994638-af22-4f13-94ee-505ee83fb2cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:04.524438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:04.525511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:04.527469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 277/548] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:04.535641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81444bb7-4373-482f-959a-a98396b7fe0c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:04.700865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81444bb7-4373-482f-959a-a98396b7fe0c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:04.737516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30821c-8e83-4494-9b7b-d5f73109cfc7 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:04.737895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30821c-8e83-4494-9b7b-d5f73109cfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:04.738088 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30821c-8e83-4494-9b7b-d5f73109cfc7 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:04.739719 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 278/549] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:04.748302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeba5e7-36dd-473f-9f77-17821a089bc0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:04.748614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeba5e7-36dd-473f-9f77-17821a089bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:04.748879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeba5e7-36dd-473f-9f77-17821a089bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:04.749763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6828a2c-0957-4531-8736-23a2997e292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:04.750438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6828a2c-0957-4531-8736-23a2997e292b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:04.750722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6828a2c-0957-4531-8736-23a2997e292b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:04.750997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6828a2c-0957-4531-8736-23a2997e292b None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:04.751840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6828a2c-0957-4531-8736-23a2997e292b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:04.752147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6828a2c-0957-4531-8736-23a2997e292b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:04.753663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6828a2c-0957-4531-8736-23a2997e292b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:04.773755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 272/550] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jun 21 18:50:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:04.781663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-efeba5e7-36dd-473f-9f77-17821a089bc0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:04.783208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-efeba5e7-36dd-473f-9f77-17821a089bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:04.787372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6828a2c-0957-4531-8736-23a2997e292b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:04.869176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:04.894913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 279/551] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:04.923730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6828a2c-0957-4531-8736-23a2997e292b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:04.933061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeba5e7-36dd-473f-9f77-17821a089bc0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:04.959751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbb61a-2d28-400a-8aa1-8351aa241691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:04.960406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbb61a-2d28-400a-8aa1-8351aa241691 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:04.960690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbb61a-2d28-400a-8aa1-8351aa241691 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:04.960968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbb61a-2d28-400a-8aa1-8351aa241691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:05.058417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:05.064897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbb61a-2d28-400a-8aa1-8351aa241691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:05.070471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbb61a-2d28-400a-8aa1-8351aa241691 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:05.074929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeba5e7-36dd-473f-9f77-17821a089bc0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:05.112451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86309-8dd5-478d-9655-378b3b2105af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:05.113392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86309-8dd5-478d-9655-378b3b2105af None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:05.113747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86309-8dd5-478d-9655-378b3b2105af None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:05.114102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86309-8dd5-478d-9655-378b3b2105af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:05.126394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce86309-8dd5-478d-9655-378b3b2105af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-cce86309-8dd5-478d-9655-378b3b2105af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:05.134716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:05.136266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:05.137608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 280/552] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:05.144094 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86309-8dd5-478d-9655-378b3b2105af None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:05.173532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:05.224745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 273/553] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:05.230252 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbb61a-2d28-400a-8aa1-8351aa241691 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:05.358163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86309-8dd5-478d-9655-378b3b2105af None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:05.427551 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c8037-98d0-42bd-a433-274efbe41d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:05.428244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c8037-98d0-42bd-a433-274efbe41d51 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:05.428532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c8037-98d0-42bd-a433-274efbe41d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:05.428814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c8037-98d0-42bd-a433-274efbe41d51 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:05.429619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c8037-98d0-42bd-a433-274efbe41d51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:05.429937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c8037-98d0-42bd-a433-274efbe41d51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:05.431374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c8037-98d0-42bd-a433-274efbe41d51 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:05.441045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbb61a-2d28-400a-8aa1-8351aa241691 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:05.458507 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 281/554] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:05.481823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c8037-98d0-42bd-a433-274efbe41d51 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:05.497440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:05.514829 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeacccd-5896-42ac-866f-0f65c839fe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:05.515639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeacccd-5896-42ac-866f-0f65c839fe4f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:05.515923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeacccd-5896-42ac-866f-0f65c839fe4f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:05.516200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeacccd-5896-42ac-866f-0f65c839fe4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:05.534689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeacccd-5896-42ac-866f-0f65c839fe4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:05.536184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeacccd-5896-42ac-866f-0f65c839fe4f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-eaeacccd-5896-42ac-866f-0f65c839fe4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:05.568662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:05.569731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:05.570597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:05.570597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:05.570597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:05.570597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:05.581828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 274/555] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:05.582691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeacccd-5896-42ac-866f-0f65c839fe4f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:05.602930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:05.643597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c8037-98d0-42bd-a433-274efbe41d51 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:05.680962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a142f-df65-4aff-be8b-08b1f7f8e716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:05.681885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a142f-df65-4aff-be8b-08b1f7f8e716 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:05.682175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a142f-df65-4aff-be8b-08b1f7f8e716 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:05.682485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a142f-df65-4aff-be8b-08b1f7f8e716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:05.695236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a142f-df65-4aff-be8b-08b1f7f8e716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:05.696812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a142f-df65-4aff-be8b-08b1f7f8e716 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:05.700415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeacccd-5896-42ac-866f-0f65c839fe4f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-443a142f-df65-4aff-be8b-08b1f7f8e716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:05.715911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:05.716838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:05.718257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:05.718257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:05.718257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:05.718257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:05.718257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 282/556] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:05.719016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9322c8f-550a-4258-833d-667ef2ec06b7 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:05.719182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9322c8f-550a-4258-833d-667ef2ec06b7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:05.721418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9322c8f-550a-4258-833d-667ef2ec06b7 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:05.721418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 283/557] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:05.722583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9322c8f-550a-4258-833d-667ef2ec06b7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:05.740880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5d789-91e6-4467-9fdb-32906b56846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:05.741629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d789-91e6-4467-9fdb-32906b56846b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:05.741920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d789-91e6-4467-9fdb-32906b56846b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:05.742254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d789-91e6-4467-9fdb-32906b56846b None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:05.743044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5d789-91e6-4467-9fdb-32906b56846b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:05.743356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5d789-91e6-4467-9fdb-32906b56846b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:05.745065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5d789-91e6-4467-9fdb-32906b56846b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:05.762745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 275/558] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:05.764150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74df41b-232e-4d56-ab32-5fdb3ce57536 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:05.764451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74df41b-232e-4d56-ab32-5fdb3ce57536 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:05.764731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74df41b-232e-4d56-ab32-5fdb3ce57536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:05.782364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a74df41b-232e-4d56-ab32-5fdb3ce57536 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:05.783659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a74df41b-232e-4d56-ab32-5fdb3ce57536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:05.803873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:05.837844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9322c8f-550a-4258-833d-667ef2ec06b7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:05.849224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:05.858975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 276/559] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:05.860101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74df41b-232e-4d56-ab32-5fdb3ce57536 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:05.903388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdf0f7-1b46-45f0-ad76-53fc4897e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:05.904504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdf0f7-1b46-45f0-ad76-53fc4897e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:05.904796 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdf0f7-1b46-45f0-ad76-53fc4897e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:05.905414 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdf0f7-1b46-45f0-ad76-53fc4897e3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:05.906165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdf0f7-1b46-45f0-ad76-53fc4897e3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:05.906745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdf0f7-1b46-45f0-ad76-53fc4897e3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:05.908823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdf0f7-1b46-45f0-ad76-53fc4897e3f5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:05.925262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 284/560] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:05.926218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdf0f7-1b46-45f0-ad76-53fc4897e3f5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:05.994698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74df41b-232e-4d56-ab32-5fdb3ce57536 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.031548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e8389-1be8-4321-971d-797c568afa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.032343 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e8389-1be8-4321-971d-797c568afa37 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.032635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e8389-1be8-4321-971d-797c568afa37 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.032915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e8389-1be8-4321-971d-797c568afa37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.039904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e8389-1be8-4321-971d-797c568afa37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:06.041400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e8389-1be8-4321-971d-797c568afa37 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:06.043333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdf0f7-1b46-45f0-ad76-53fc4897e3f5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-163e8389-1be8-4321-971d-797c568afa37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:06.055367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:06.056465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:06.057518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:06.057518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:06.057518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:06.057518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:06.057518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:06.057518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:06.057518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:06.057518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:06.057518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 277/561] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:06.058023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e8389-1be8-4321-971d-797c568afa37 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:06.071897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:06.089787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df589-08f6-4f23-a1e9-445046c6a9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.090604 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df589-08f6-4f23-a1e9-445046c6a9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.090967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df589-08f6-4f23-a1e9-445046c6a9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.091346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df589-08f6-4f23-a1e9-445046c6a9b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.102608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319df589-08f6-4f23-a1e9-445046c6a9b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-319df589-08f6-4f23-a1e9-445046c6a9b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:06.110581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:06.111794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:06.112702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 285/562] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:06.113821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd3508-608e-48ea-a7df-30b3af628883 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.114109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd3508-608e-48ea-a7df-30b3af628883 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.114388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd3508-608e-48ea-a7df-30b3af628883 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.119808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 286/563] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:06.119808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd3508-608e-48ea-a7df-30b3af628883 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:06.131697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:06.178933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e8389-1be8-4321-971d-797c568afa37 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.215753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f481e7-b21b-489d-8e7c-9dc774bf5062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.216655 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f481e7-b21b-489d-8e7c-9dc774bf5062 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.217014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f481e7-b21b-489d-8e7c-9dc774bf5062 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.217394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f481e7-b21b-489d-8e7c-9dc774bf5062 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.225664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f481e7-b21b-489d-8e7c-9dc774bf5062 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:06.227219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f481e7-b21b-489d-8e7c-9dc774bf5062 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-34f481e7-b21b-489d-8e7c-9dc774bf5062 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:06.242088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:06.243116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:06.244065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:06.244597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd3508-608e-48ea-a7df-30b3af628883 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.248101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 278/564] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:06.249000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f481e7-b21b-489d-8e7c-9dc774bf5062 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:06.284672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c91755-27c7-4a32-a384-b3ce5c09e52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.285414 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c91755-27c7-4a32-a384-b3ce5c09e52d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.285703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c91755-27c7-4a32-a384-b3ce5c09e52d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.285984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c91755-27c7-4a32-a384-b3ce5c09e52d None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.286891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c91755-27c7-4a32-a384-b3ce5c09e52d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:06.287211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c91755-27c7-4a32-a384-b3ce5c09e52d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:06.288725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c91755-27c7-4a32-a384-b3ce5c09e52d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:06.301309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 287/565] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:06.302549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c25873-d7a4-4467-9e18-81c173b4fd2f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.302845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c25873-d7a4-4467-9e18-81c173b4fd2f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.303122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c25873-d7a4-4467-9e18-81c173b4fd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.320617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-05c25873-d7a4-4467-9e18-81c173b4fd2f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:06.324868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-05c25873-d7a4-4467-9e18-81c173b4fd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:06.360325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:06.371837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f481e7-b21b-489d-8e7c-9dc774bf5062 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.390876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:06.405119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 288/566] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:06.405119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c25873-d7a4-4467-9e18-81c173b4fd2f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:06.425494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d292ee-5b29-4c6b-9126-bbec37f93ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.426199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d292ee-5b29-4c6b-9126-bbec37f93ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.426488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d292ee-5b29-4c6b-9126-bbec37f93ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.426763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d292ee-5b29-4c6b-9126-bbec37f93ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.427514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d292ee-5b29-4c6b-9126-bbec37f93ca8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:06.427824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d292ee-5b29-4c6b-9126-bbec37f93ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:06.429222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d292ee-5b29-4c6b-9126-bbec37f93ca8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:06.438463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 279/567] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:06.439280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d292ee-5b29-4c6b-9126-bbec37f93ca8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:06.498533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c25873-d7a4-4467-9e18-81c173b4fd2f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.532545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7564035-0bd7-4872-ab0d-1d74704720f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.533257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7564035-0bd7-4872-ab0d-1d74704720f4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.533544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7564035-0bd7-4872-ab0d-1d74704720f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.533827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7564035-0bd7-4872-ab0d-1d74704720f4 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.534662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7564035-0bd7-4872-ab0d-1d74704720f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:06.534971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7564035-0bd7-4872-ab0d-1d74704720f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:06.536399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7564035-0bd7-4872-ab0d-1d74704720f4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:06.538637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d292ee-5b29-4c6b-9126-bbec37f93ca8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.554122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 289/568] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:06.555515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b56bb-b063-4a13-9048-90b6e935a51f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.555809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b56bb-b063-4a13-9048-90b6e935a51f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.556087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b56bb-b063-4a13-9048-90b6e935a51f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.557364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 290/569] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:06.558601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b56bb-b063-4a13-9048-90b6e935a51f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:06.577535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22038e1-abcb-4e10-9a3a-4e87bd4b6bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.578187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22038e1-abcb-4e10-9a3a-4e87bd4b6bec None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.578467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22038e1-abcb-4e10-9a3a-4e87bd4b6bec None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.578746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22038e1-abcb-4e10-9a3a-4e87bd4b6bec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.579571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22038e1-abcb-4e10-9a3a-4e87bd4b6bec None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:06.579885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22038e1-abcb-4e10-9a3a-4e87bd4b6bec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:06.581402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22038e1-abcb-4e10-9a3a-4e87bd4b6bec None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:06.587031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 280/570] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/role_assignments?user.id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:06.587818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22038e1-abcb-4e10-9a3a-4e87bd4b6bec None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:06.587959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:06.648471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:06.680225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b56bb-b063-4a13-9048-90b6e935a51f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.715728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22038e1-abcb-4e10-9a3a-4e87bd4b6bec None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.720025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b28e9d-2cee-4807-9db0-f16ea4af8b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.720843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b28e9d-2cee-4807-9db0-f16ea4af8b72 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.721151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b28e9d-2cee-4807-9db0-f16ea4af8b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.721433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b28e9d-2cee-4807-9db0-f16ea4af8b72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.734735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b28e9d-2cee-4807-9db0-f16ea4af8b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e4b28e9d-2cee-4807-9db0-f16ea4af8b72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:06.742671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:06.743934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:06.744884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 291/571] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:06.746204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b28e9d-2cee-4807-9db0-f16ea4af8b72 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:06.761460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa02ad0-14bd-4db0-b806-8d32f5c5b924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.762039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa02ad0-14bd-4db0-b806-8d32f5c5b924 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.762886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa02ad0-14bd-4db0-b806-8d32f5c5b924 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.762886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa02ad0-14bd-4db0-b806-8d32f5c5b924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.774193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa02ad0-14bd-4db0-b806-8d32f5c5b924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:06.775842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa02ad0-14bd-4db0-b806-8d32f5c5b924 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-bfa02ad0-14bd-4db0-b806-8d32f5c5b924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:06.795001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:06.796121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:06.797130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:06.797130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:06.797130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:06.797130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:06.797130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 281/572] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:06.797454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60f45f-9036-4faa-8fa8-b4e86e5a82a4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.797740 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60f45f-9036-4faa-8fa8-b4e86e5a82a4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.798016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60f45f-9036-4faa-8fa8-b4e86e5a82a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.815484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1c60f45f-9036-4faa-8fa8-b4e86e5a82a4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:06.816682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1c60f45f-9036-4faa-8fa8-b4e86e5a82a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:06.855384 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:06.869934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b28e9d-2cee-4807-9db0-f16ea4af8b72 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:06.897119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 282/573] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:06.897119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60f45f-9036-4faa-8fa8-b4e86e5a82a4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:06.912981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae5bac-10d3-4fb2-bca7-765b421e6106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:06.913730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae5bac-10d3-4fb2-bca7-765b421e6106 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:06.914115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae5bac-10d3-4fb2-bca7-765b421e6106 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:06.914397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae5bac-10d3-4fb2-bca7-765b421e6106 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:06.927266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae5bac-10d3-4fb2-bca7-765b421e6106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:06.928139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:06.928719 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae5bac-10d3-4fb2-bca7-765b421e6106 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b0ae5bac-10d3-4fb2-bca7-765b421e6106 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:06.947875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:06.949064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:06.950010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:06.950010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:06.950010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:06.950010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:06.950010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 292/574] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:06.950010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae5bac-10d3-4fb2-bca7-765b421e6106 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.020718 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60f45f-9036-4faa-8fa8-b4e86e5a82a4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.062571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe50ec9-7e71-4bad-a6a2-f4da9488aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.063550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe50ec9-7e71-4bad-a6a2-f4da9488aeff None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.064020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe50ec9-7e71-4bad-a6a2-f4da9488aeff None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.064481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe50ec9-7e71-4bad-a6a2-f4da9488aeff None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.064909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae5bac-10d3-4fb2-bca7-765b421e6106 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.065674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe50ec9-7e71-4bad-a6a2-f4da9488aeff None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:07.066171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe50ec9-7e71-4bad-a6a2-f4da9488aeff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.067846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe50ec9-7e71-4bad-a6a2-f4da9488aeff None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.076683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 283/575] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:07.077640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe50ec9-7e71-4bad-a6a2-f4da9488aeff None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.107757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb53baf-660a-4af5-9397-565910f9b370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.108458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb53baf-660a-4af5-9397-565910f9b370 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.108742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb53baf-660a-4af5-9397-565910f9b370 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.109019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb53baf-660a-4af5-9397-565910f9b370 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.109771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb53baf-660a-4af5-9397-565910f9b370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:07.110076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb53baf-660a-4af5-9397-565910f9b370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.111541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb53baf-660a-4af5-9397-565910f9b370 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.120884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:07.129718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 293/576] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:07.129718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb53baf-660a-4af5-9397-565910f9b370 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.186389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:07.210146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe50ec9-7e71-4bad-a6a2-f4da9488aeff None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.235654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb53baf-660a-4af5-9397-565910f9b370 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.251470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776d761-01cb-49f3-99ac-7da18c4710e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.252271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776d761-01cb-49f3-99ac-7da18c4710e1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.252563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776d761-01cb-49f3-99ac-7da18c4710e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.252848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776d761-01cb-49f3-99ac-7da18c4710e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.264627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776d761-01cb-49f3-99ac-7da18c4710e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:07.271480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef319f-0662-4791-84ed-cbc85eeed619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.272145 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef319f-0662-4791-84ed-cbc85eeed619 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.272431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef319f-0662-4791-84ed-cbc85eeed619 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.272709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef319f-0662-4791-84ed-cbc85eeed619 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-1776d761-01cb-49f3-99ac-7da18c4710e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:07.273406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:07.274306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:07.275123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:07.275744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef319f-0662-4791-84ed-cbc85eeed619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:07.275744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef319f-0662-4791-84ed-cbc85eeed619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.275744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef319f-0662-4791-84ed-cbc85eeed619 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.280440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 284/577] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:07.281393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776d761-01cb-49f3-99ac-7da18c4710e1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.294013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 294/578] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:07.294013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef319f-0662-4791-84ed-cbc85eeed619 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.361714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:07.397717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776d761-01cb-49f3-99ac-7da18c4710e1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.406805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef319f-0662-4791-84ed-cbc85eeed619 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.433972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eadf12-81fd-487e-add9-06164289ce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.434772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eadf12-81fd-487e-add9-06164289ce44 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.435066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eadf12-81fd-487e-add9-06164289ce44 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.435349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eadf12-81fd-487e-add9-06164289ce44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.445539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633da7ad-f908-4d68-a822-d67d81e6efff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.446416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da7ad-f908-4d68-a822-d67d81e6efff None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.446701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da7ad-f908-4d68-a822-d67d81e6efff None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.446801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eadf12-81fd-487e-add9-06164289ce44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.446986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da7ad-f908-4d68-a822-d67d81e6efff None admin] PATH_INFO: `/v3/users/glance` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.448749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eadf12-81fd-487e-add9-06164289ce44 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.455257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:07.456668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633da7ad-f908-4d68-a822-d67d81e6efff None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.458184 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633da7ad-f908-4d68-a822-d67d81e6efff None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-57eadf12-81fd-487e-add9-06164289ce44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:07.468666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:07.469867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:07.470770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:07.470770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:07.470770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:07.470770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:07.470770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 285/579] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:07.471054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eadf12-81fd-487e-add9-06164289ce44 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-633da7ad-f908-4d68-a822-d67d81e6efff None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:07.475389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:07.476531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:50:07.477748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:07.478980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 295/580] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:07.479864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633da7ad-f908-4d68-a822-d67d81e6efff None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.577119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eadf12-81fd-487e-add9-06164289ce44 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.590231 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633da7ad-f908-4d68-a822-d67d81e6efff None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.611684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe2e9-0867-4a10-9eac-0a03e0c9fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.612309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:07.612426 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe2e9-0867-4a10-9eac-0a03e0c9fc43 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.612651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe2e9-0867-4a10-9eac-0a03e0c9fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.613298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe2e9-0867-4a10-9eac-0a03e0c9fc43 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.613728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe2e9-0867-4a10-9eac-0a03e0c9fc43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:07.614077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe2e9-0867-4a10-9eac-0a03e0c9fc43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.615707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe2e9-0867-4a10-9eac-0a03e0c9fc43 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.628369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 286/581] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:07.628369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe2e9-0867-4a10-9eac-0a03e0c9fc43 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.633383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a93f45-d96a-4a20-bccb-185975d6aa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.634348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a93f45-d96a-4a20-bccb-185975d6aa8c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.634710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a93f45-d96a-4a20-bccb-185975d6aa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.635062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a93f45-d96a-4a20-bccb-185975d6aa8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.644566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a93f45-d96a-4a20-bccb-185975d6aa8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.646424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a93f45-d96a-4a20-bccb-185975d6aa8c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-63a93f45-d96a-4a20-bccb-185975d6aa8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:07.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:07.664291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:07.665332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:07.665332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:07.665332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:07.665332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:07.665332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:07.665332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:07.665332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:07.665332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:07.665332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 296/582] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:07.665928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a93f45-d96a-4a20-bccb-185975d6aa8c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.695892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:07.745165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe2e9-0867-4a10-9eac-0a03e0c9fc43 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.772145 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a93f45-d96a-4a20-bccb-185975d6aa8c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.783212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffb756-304b-4586-ad49-852ac7cd40b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.784015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffb756-304b-4586-ad49-852ac7cd40b2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.784407 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffb756-304b-4586-ad49-852ac7cd40b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.784792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffb756-304b-4586-ad49-852ac7cd40b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.785779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffb756-304b-4586-ad49-852ac7cd40b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:07.786302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffb756-304b-4586-ad49-852ac7cd40b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.787854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffb756-304b-4586-ad49-852ac7cd40b2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.805941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 287/583] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:07.805941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffb756-304b-4586-ad49-852ac7cd40b2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.811399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8b7da-9a6b-4fcb-980b-2e9ef6c5f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.812148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8b7da-9a6b-4fcb-980b-2e9ef6c5f72b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.812780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8b7da-9a6b-4fcb-980b-2e9ef6c5f72b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.813132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8b7da-9a6b-4fcb-980b-2e9ef6c5f72b None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.814082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8b7da-9a6b-4fcb-980b-2e9ef6c5f72b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:07.814491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8b7da-9a6b-4fcb-980b-2e9ef6c5f72b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.816009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8b7da-9a6b-4fcb-980b-2e9ef6c5f72b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.838181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 297/584] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:07.838181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8b7da-9a6b-4fcb-980b-2e9ef6c5f72b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:07.878760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:07.912842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffb756-304b-4586-ad49-852ac7cd40b2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.934305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:07.947019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8b7da-9a6b-4fcb-980b-2e9ef6c5f72b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:07.951600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452059ac-3671-493f-abe8-8b75892c62cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.952603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452059ac-3671-493f-abe8-8b75892c62cc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.953041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452059ac-3671-493f-abe8-8b75892c62cc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.953488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452059ac-3671-493f-abe8-8b75892c62cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.965158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452059ac-3671-493f-abe8-8b75892c62cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.966850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452059ac-3671-493f-abe8-8b75892c62cc None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.984183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b0ddf-7b4c-4661-81c9-9bf25d34c0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:07.984843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b0ddf-7b4c-4661-81c9-9bf25d34c0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:07.985150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b0ddf-7b4c-4661-81c9-9bf25d34c0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:07.985435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b0ddf-7b4c-4661-81c9-9bf25d34c0b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:07.986226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b0ddf-7b4c-4661-81c9-9bf25d34c0b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:07.986538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b0ddf-7b4c-4661-81c9-9bf25d34c0b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-452059ac-3671-493f-abe8-8b75892c62cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:07.986824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:07.987872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:07.988961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:07.988961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:07.988961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:07.988961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:07.989259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b0ddf-7b4c-4661-81c9-9bf25d34c0b6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:07.990644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 288/585] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:07.991590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452059ac-3671-493f-abe8-8b75892c62cc None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.004769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 298/586] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:08.005640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b0ddf-7b4c-4661-81c9-9bf25d34c0b6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.108809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452059ac-3671-493f-abe8-8b75892c62cc None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.110681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:08.116015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b0ddf-7b4c-4661-81c9-9bf25d34c0b6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.147575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d9808-2199-4112-a461-94a3a19f4033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.148375 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d9808-2199-4112-a461-94a3a19f4033 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.148664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d9808-2199-4112-a461-94a3a19f4033 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.148943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d9808-2199-4112-a461-94a3a19f4033 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.154315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96264968-4d93-4a16-b691-9a3377338cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.155113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96264968-4d93-4a16-b691-9a3377338cee None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.155399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96264968-4d93-4a16-b691-9a3377338cee None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.155675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96264968-4d93-4a16-b691-9a3377338cee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.156961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d9808-2199-4112-a461-94a3a19f4033 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.158484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d9808-2199-4112-a461-94a3a19f4033 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.168111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96264968-4d93-4a16-b691-9a3377338cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.169759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96264968-4d93-4a16-b691-9a3377338cee None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d44d9808-2199-4112-a461-94a3a19f4033 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:08.175125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:08.176212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:50:08.177411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:08.181136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 289/587] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:08.181796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d9808-2199-4112-a461-94a3a19f4033 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-96264968-4d93-4a16-b691-9a3377338cee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:08.193828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:08.194943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:08.196014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:08.196014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:08.196014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:08.196014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:08.200914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 299/588] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:08.201651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96264968-4d93-4a16-b691-9a3377338cee None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.218678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:08.298622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d9808-2199-4112-a461-94a3a19f4033 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.316843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96264968-4d93-4a16-b691-9a3377338cee None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.339892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e1ce3-7d9e-4750-8164-3e28bdbe93e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.340800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e1ce3-7d9e-4750-8164-3e28bdbe93e6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.341126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e1ce3-7d9e-4750-8164-3e28bdbe93e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.341408 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e1ce3-7d9e-4750-8164-3e28bdbe93e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.342153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e1ce3-7d9e-4750-8164-3e28bdbe93e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:08.342472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e1ce3-7d9e-4750-8164-3e28bdbe93e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.343865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e1ce3-7d9e-4750-8164-3e28bdbe93e6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.357072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 290/589] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:08.357958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e1ce3-7d9e-4750-8164-3e28bdbe93e6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.364988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e688c42-69e4-4ba2-a897-69330d362496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.365923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e688c42-69e4-4ba2-a897-69330d362496 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.366206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e688c42-69e4-4ba2-a897-69330d362496 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.366485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e688c42-69e4-4ba2-a897-69330d362496 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/dcbcec65220c4ebb95f974925fa73f34/roles/37aa79fddde643aeab12191e2c79681f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.382492 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e688c42-69e4-4ba2-a897-69330d362496 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=dcbcec65220c4ebb95f974925fa73f34, role_id=37aa79fddde643aeab12191e2c79681f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.383930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e688c42-69e4-4ba2-a897-69330d362496 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.393449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:08.399700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 300/590] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:50:08 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/dcbcec65220c4ebb95f974925fa73f34/roles/37aa79fddde643aeab12191e2c79681f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:50:08.401228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e688c42-69e4-4ba2-a897-69330d362496 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.488039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:08.499430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e1ce3-7d9e-4750-8164-3e28bdbe93e6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.536143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e688c42-69e4-4ba2-a897-69330d362496 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.537775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b731af9-c9db-40ff-8947-fbeae590e384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.538636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b731af9-c9db-40ff-8947-fbeae590e384 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.539100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b731af9-c9db-40ff-8947-fbeae590e384 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.539573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b731af9-c9db-40ff-8947-fbeae590e384 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.540632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b731af9-c9db-40ff-8947-fbeae590e384 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:08.541201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b731af9-c9db-40ff-8947-fbeae590e384 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.543469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b731af9-c9db-40ff-8947-fbeae590e384 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.565399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 291/591] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jun 21 18:50:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:08.566615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b731af9-c9db-40ff-8947-fbeae590e384 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.582009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc9364-e5c8-440f-9a68-935bdbe78ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.582687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc9364-e5c8-440f-9a68-935bdbe78ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.582981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc9364-e5c8-440f-9a68-935bdbe78ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.583259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc9364-e5c8-440f-9a68-935bdbe78ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.583997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc9364-e5c8-440f-9a68-935bdbe78ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:08.584310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc9364-e5c8-440f-9a68-935bdbe78ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.585902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc9364-e5c8-440f-9a68-935bdbe78ad8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.603412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 301/592] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:08.605111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc9364-e5c8-440f-9a68-935bdbe78ad8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.680724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:08.702284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b731af9-c9db-40ff-8947-fbeae590e384 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.726322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc9364-e5c8-440f-9a68-935bdbe78ad8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.740812 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514de6-f74f-4981-a783-0ec4d7042d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.741942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514de6-f74f-4981-a783-0ec4d7042d6e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.742461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514de6-f74f-4981-a783-0ec4d7042d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.742953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514de6-f74f-4981-a783-0ec4d7042d6e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.751427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e514de6-f74f-4981-a783-0ec4d7042d6e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.753218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e514de6-f74f-4981-a783-0ec4d7042d6e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.761758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fbbb8-c480-4d5d-905f-46d8c16ddb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.762885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fbbb8-c480-4d5d-905f-46d8c16ddb6c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.763374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fbbb8-c480-4d5d-905f-46d8c16ddb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.763853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fbbb8-c480-4d5d-905f-46d8c16ddb6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-9e514de6-f74f-4981-a783-0ec4d7042d6e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:08.768437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:08.769532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:08.770489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 292/593] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:08.771072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514de6-f74f-4981-a783-0ec4d7042d6e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.780738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fbbb8-c480-4d5d-905f-46d8c16ddb6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.782527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fbbb8-c480-4d5d-905f-46d8c16ddb6c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.784871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-739fbbb8-c480-4d5d-905f-46d8c16ddb6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:08.803548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:08.804513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:08.805452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:08.805452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:08.805452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:08.805452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:08.811446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 302/594] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:08.811446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620faed-2f95-465c-8e01-cf9d8b02529c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.811446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620faed-2f95-465c-8e01-cf9d8b02529c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.811446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620faed-2f95-465c-8e01-cf9d8b02529c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.811933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 303/595] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:08.813030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620faed-2f95-465c-8e01-cf9d8b02529c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.888640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514de6-f74f-4981-a783-0ec4d7042d6e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.925801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620faed-2f95-465c-8e01-cf9d8b02529c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:08.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd120fa-87f7-4452-8015-fa7bb4bb6683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.936414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd120fa-87f7-4452-8015-fa7bb4bb6683 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.936700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd120fa-87f7-4452-8015-fa7bb4bb6683 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.937676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd120fa-87f7-4452-8015-fa7bb4bb6683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.942815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:08.946080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd120fa-87f7-4452-8015-fa7bb4bb6683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.947588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd120fa-87f7-4452-8015-fa7bb4bb6683 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-1cd120fa-87f7-4452-8015-fa7bb4bb6683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:08.962709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:08.963709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:08.964604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:08.964604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:08.964604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:08.964604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:08.964604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:08.964604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:08.964604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:08.964604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:08.964604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 293/596] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:08.965751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd120fa-87f7-4452-8015-fa7bb4bb6683 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:08.971475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1585f3a-3b9c-41da-9215-4320f009edf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:08.972152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1585f3a-3b9c-41da-9215-4320f009edf3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:08.972434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1585f3a-3b9c-41da-9215-4320f009edf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:08.972709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1585f3a-3b9c-41da-9215-4320f009edf3 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:08.973586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1585f3a-3b9c-41da-9215-4320f009edf3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:08.973895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1585f3a-3b9c-41da-9215-4320f009edf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:08.975341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1585f3a-3b9c-41da-9215-4320f009edf3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:08.995739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 304/597] 10.222.0.70 () {56 vars in 1139 bytes} [Tue Jun 21 18:50:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:08.999319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78142a97-4487-415a-9243-a70ab2060974 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.000288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78142a97-4487-415a-9243-a70ab2060974 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.001158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78142a97-4487-415a-9243-a70ab2060974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.019228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-78142a97-4487-415a-9243-a70ab2060974 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:09.020608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-78142a97-4487-415a-9243-a70ab2060974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:09.041147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:09.076247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd120fa-87f7-4452-8015-fa7bb4bb6683 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:09.100652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 305/598] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:09.101705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78142a97-4487-415a-9243-a70ab2060974 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.121382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca8a60-415b-43d1-84e8-d5f40d873f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:09.122262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca8a60-415b-43d1-84e8-d5f40d873f6c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.122646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca8a60-415b-43d1-84e8-d5f40d873f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.123025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca8a60-415b-43d1-84e8-d5f40d873f6c None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.124003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca8a60-415b-43d1-84e8-d5f40d873f6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:09.124451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca8a60-415b-43d1-84e8-d5f40d873f6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:09.126142 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca8a60-415b-43d1-84e8-d5f40d873f6c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:09.138742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 294/599] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:09.139769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca8a60-415b-43d1-84e8-d5f40d873f6c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.215798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:09.218782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78142a97-4487-415a-9243-a70ab2060974 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:09.251761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca8a60-415b-43d1-84e8-d5f40d873f6c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:09.261139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03013903-5343-4556-b1b2-5b57f8c430ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:09.261834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03013903-5343-4556-b1b2-5b57f8c430ad None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.262118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03013903-5343-4556-b1b2-5b57f8c430ad None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.262397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03013903-5343-4556-b1b2-5b57f8c430ad None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.263246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03013903-5343-4556-b1b2-5b57f8c430ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:09.263558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03013903-5343-4556-b1b2-5b57f8c430ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:09.265000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03013903-5343-4556-b1b2-5b57f8c430ad None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:09.277834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 306/600] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:09.278659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03013903-5343-4556-b1b2-5b57f8c430ad None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.288454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:09.295404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac9514-da35-4b24-83a1-593009f9c4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:09.296203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac9514-da35-4b24-83a1-593009f9c4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.296489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac9514-da35-4b24-83a1-593009f9c4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.296756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac9514-da35-4b24-83a1-593009f9c4b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.308335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac9514-da35-4b24-83a1-593009f9c4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:09.309838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac9514-da35-4b24-83a1-593009f9c4b0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-24ac9514-da35-4b24-83a1-593009f9c4b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:09.332936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:09.333997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:09.334897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:09.334897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:09.334897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:09.334897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:09.334897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 295/601] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:09.340145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac9514-da35-4b24-83a1-593009f9c4b0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.393346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03013903-5343-4556-b1b2-5b57f8c430ad None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:09.428092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b259a3-f39e-46b4-8007-87602c93ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:09.429204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b259a3-f39e-46b4-8007-87602c93ed48 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.430080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b259a3-f39e-46b4-8007-87602c93ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.430522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b259a3-f39e-46b4-8007-87602c93ed48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.442921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac9514-da35-4b24-83a1-593009f9c4b0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:09.443319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b259a3-f39e-46b4-8007-87602c93ed48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-94b259a3-f39e-46b4-8007-87602c93ed48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:09.451866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:09.453014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:09.454057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:09.456098 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 307/602] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:09.457002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b259a3-f39e-46b4-8007-87602c93ed48 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.483510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:09.485150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f20c7-db94-437d-b9a0-0769b4206419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:09.485947 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f20c7-db94-437d-b9a0-0769b4206419 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.486233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f20c7-db94-437d-b9a0-0769b4206419 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.486506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f20c7-db94-437d-b9a0-0769b4206419 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.493872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f20c7-db94-437d-b9a0-0769b4206419 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:09.495869 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f20c7-db94-437d-b9a0-0769b4206419 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ac9f20c7-db94-437d-b9a0-0769b4206419 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:09.510030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:09.511707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:09.512816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:09.512816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:09.512816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:09.512816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:09.512816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:09.512816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:09.512816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:09.512816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:09.516576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 296/603] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:09.517488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f20c7-db94-437d-b9a0-0769b4206419 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.564559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:09.579439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b259a3-f39e-46b4-8007-87602c93ed48 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:09.616754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6459-dcb0-4785-b21d-e02ac070d2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:09.617675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6459-dcb0-4785-b21d-e02ac070d2b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.617955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6459-dcb0-4785-b21d-e02ac070d2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.618234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6459-dcb0-4785-b21d-e02ac070d2b7 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/915704f26c814eaa9e6c2a68533a9c66/roles/37aa79fddde643aeab12191e2c79681f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.628289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f20c7-db94-437d-b9a0-0769b4206419 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:09.634134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6459-dcb0-4785-b21d-e02ac070d2b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=915704f26c814eaa9e6c2a68533a9c66, role_id=37aa79fddde643aeab12191e2c79681f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:09.636225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6459-dcb0-4785-b21d-e02ac070d2b7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:09.657522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 308/604] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:50:09 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/915704f26c814eaa9e6c2a68533a9c66/roles/37aa79fddde643aeab12191e2c79681f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:50:09.658705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6459-dcb0-4785-b21d-e02ac070d2b7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.672201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196084b-8ce5-4c70-89b3-3e5c6c0936d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:09.673433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196084b-8ce5-4c70-89b3-3e5c6c0936d6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.674035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196084b-8ce5-4c70-89b3-3e5c6c0936d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.675092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196084b-8ce5-4c70-89b3-3e5c6c0936d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.676587 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196084b-8ce5-4c70-89b3-3e5c6c0936d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:09.677399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196084b-8ce5-4c70-89b3-3e5c6c0936d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:09.679577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196084b-8ce5-4c70-89b3-3e5c6c0936d6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:09.701134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 297/605] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:09.702140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196084b-8ce5-4c70-89b3-3e5c6c0936d6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.763855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:09.802805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6459-dcb0-4785-b21d-e02ac070d2b7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:09.818079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196084b-8ce5-4c70-89b3-3e5c6c0936d6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:09.820719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:09.847737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f525a6-4f07-4f6c-b6f3-de44e1ed1c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:09.848527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f525a6-4f07-4f6c-b6f3-de44e1ed1c13 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.849178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f525a6-4f07-4f6c-b6f3-de44e1ed1c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.849178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f525a6-4f07-4f6c-b6f3-de44e1ed1c13 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.850103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f525a6-4f07-4f6c-b6f3-de44e1ed1c13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:09.850554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f525a6-4f07-4f6c-b6f3-de44e1ed1c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:09.851877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f525a6-4f07-4f6c-b6f3-de44e1ed1c13 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:09.858980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f780e1-5ec5-4972-be7d-28b65c65f30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:09.859764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f780e1-5ec5-4972-be7d-28b65c65f30a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:09.860144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f780e1-5ec5-4972-be7d-28b65c65f30a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:09.860575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 309/606] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:09.860686 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f780e1-5ec5-4972-be7d-28b65c65f30a None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:09.861281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f525a6-4f07-4f6c-b6f3-de44e1ed1c13 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.861668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f780e1-5ec5-4972-be7d-28b65c65f30a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:09.862034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f780e1-5ec5-4972-be7d-28b65c65f30a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:09.865859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f780e1-5ec5-4972-be7d-28b65c65f30a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:09.888851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 298/607] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:09.890314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f780e1-5ec5-4972-be7d-28b65c65f30a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:09.973318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f525a6-4f07-4f6c-b6f3-de44e1ed1c13 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:10.008100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:10.017446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296442ec-5258-4cad-9131-30501039b99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:10.018168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296442ec-5258-4cad-9131-30501039b99d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.018455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296442ec-5258-4cad-9131-30501039b99d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.018734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296442ec-5258-4cad-9131-30501039b99d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.027873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f780e1-5ec5-4972-be7d-28b65c65f30a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:10.028822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296442ec-5258-4cad-9131-30501039b99d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:10.031691 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296442ec-5258-4cad-9131-30501039b99d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-296442ec-5258-4cad-9131-30501039b99d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:10.051215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:10.052409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:10.053499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:10.053499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:10.053499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:10.053499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:10.053499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:10.053499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:10.053499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:10.053499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:10.056559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 310/608] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:10.058455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296442ec-5258-4cad-9131-30501039b99d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:10.068028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3118f-3f81-4039-899c-6da35bba805d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:10.068984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3118f-3f81-4039-899c-6da35bba805d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.069431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3118f-3f81-4039-899c-6da35bba805d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.069821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3118f-3f81-4039-899c-6da35bba805d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.082991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3118f-3f81-4039-899c-6da35bba805d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:10.084779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3118f-3f81-4039-899c-6da35bba805d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-cfd3118f-3f81-4039-899c-6da35bba805d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:10.103933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:10.105064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:10.106162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:10.106162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:10.106162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:10.106162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:10.107799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 299/609] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:10.108710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3118f-3f81-4039-899c-6da35bba805d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:10.127690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:10.203338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296442ec-5258-4cad-9131-30501039b99d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:10.224857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3118f-3f81-4039-899c-6da35bba805d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:10.251102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097b432-6176-4b82-a1f8-c66ef81fc7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:10.252499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097b432-6176-4b82-a1f8-c66ef81fc7ae None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.253205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097b432-6176-4b82-a1f8-c66ef81fc7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.253627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097b432-6176-4b82-a1f8-c66ef81fc7ae None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/5b8161446e1344108f8283d46da23ab9/roles/37aa79fddde643aeab12191e2c79681f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.263872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73952e49-13c4-4cad-859b-cfd279d1fcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:10.264735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73952e49-13c4-4cad-859b-cfd279d1fcdc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.265219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73952e49-13c4-4cad-859b-cfd279d1fcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.266170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73952e49-13c4-4cad-859b-cfd279d1fcdc None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.267097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73952e49-13c4-4cad-859b-cfd279d1fcdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:10.267643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73952e49-13c4-4cad-859b-cfd279d1fcdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:10.269814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73952e49-13c4-4cad-859b-cfd279d1fcdc None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:10.273775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097b432-6176-4b82-a1f8-c66ef81fc7ae None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=5b8161446e1344108f8283d46da23ab9, role_id=37aa79fddde643aeab12191e2c79681f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:10.275382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097b432-6176-4b82-a1f8-c66ef81fc7ae None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:10.286624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 300/610] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:10.288425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73952e49-13c4-4cad-859b-cfd279d1fcdc None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:10.293373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 311/611] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:50:10 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/5b8161446e1344108f8283d46da23ab9/roles/37aa79fddde643aeab12191e2c79681f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:50:10.294081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097b432-6176-4b82-a1f8-c66ef81fc7ae None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:10.366603 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:10.443015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:10.487862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097b432-6176-4b82-a1f8-c66ef81fc7ae None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:10.493500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73952e49-13c4-4cad-859b-cfd279d1fcdc None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:10.548820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebf3c7-5b13-4508-bc2b-d5d7c674e9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:10.550289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebf3c7-5b13-4508-bc2b-d5d7c674e9b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.550821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebf3c7-5b13-4508-bc2b-d5d7c674e9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.551345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebf3c7-5b13-4508-bc2b-d5d7c674e9b4 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/808b530d24b3487b9a20ee74c131502e/roles/37aa79fddde643aeab12191e2c79681f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.553801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839dffcc-4f6d-4137-9fad-2e91a8bdb52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:10.554894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dffcc-4f6d-4137-9fad-2e91a8bdb52f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.555293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dffcc-4f6d-4137-9fad-2e91a8bdb52f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.555751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dffcc-4f6d-4137-9fad-2e91a8bdb52f None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.558866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839dffcc-4f6d-4137-9fad-2e91a8bdb52f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:10.559258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839dffcc-4f6d-4137-9fad-2e91a8bdb52f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:10.560755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839dffcc-4f6d-4137-9fad-2e91a8bdb52f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:10.571784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebf3c7-5b13-4508-bc2b-d5d7c674e9b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=808b530d24b3487b9a20ee74c131502e, role_id=37aa79fddde643aeab12191e2c79681f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:10.573344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebf3c7-5b13-4508-bc2b-d5d7c674e9b4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:10.579628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 301/612] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:10.581976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c757be5-1fdf-42f5-9e4c-86e626bef3fe None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.581976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c757be5-1fdf-42f5-9e4c-86e626bef3fe None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.581976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c757be5-1fdf-42f5-9e4c-86e626bef3fe None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.583427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 302/613] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:10.589789 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8134b5-137b-4f67-bb5d-60073abae602 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.590580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8134b5-137b-4f67-bb5d-60073abae602 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.591107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8134b5-137b-4f67-bb5d-60073abae602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.593371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 312/614] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:50:10 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/808b530d24b3487b9a20ee74c131502e/roles/37aa79fddde643aeab12191e2c79681f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:50:10.594362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebf3c7-5b13-4508-bc2b-d5d7c674e9b4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:10.614386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6b8134b5-137b-4f67-bb5d-60073abae602 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:10.616464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6b8134b5-137b-4f67-bb5d-60073abae602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:10.746188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 303/615] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:10.748973 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:10.760169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebf3c7-5b13-4508-bc2b-d5d7c674e9b4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:10.786346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8134b5-137b-4f67-bb5d-60073abae602 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:10.814049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fa4a1-9904-4994-9da9-5fecbb6e16f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:10.814871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa4a1-9904-4994-9da9-5fecbb6e16f7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.815163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa4a1-9904-4994-9da9-5fecbb6e16f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.815445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa4a1-9904-4994-9da9-5fecbb6e16f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.823645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fa4a1-9904-4994-9da9-5fecbb6e16f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:10.825146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fa4a1-9904-4994-9da9-5fecbb6e16f7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-395fa4a1-9904-4994-9da9-5fecbb6e16f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:10.843240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:10.844223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:10.845501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 313/616] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:10.850323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fa4a1-9904-4994-9da9-5fecbb6e16f7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:10.883065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:10.923946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8134b5-137b-4f67-bb5d-60073abae602 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:10.974553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed30a72-b431-4919-8471-0febebbe12ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:10.975357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed30a72-b431-4919-8471-0febebbe12ab None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:10.975644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed30a72-b431-4919-8471-0febebbe12ab None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:10.975940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed30a72-b431-4919-8471-0febebbe12ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:10.987819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed30a72-b431-4919-8471-0febebbe12ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:10.995424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fa4a1-9904-4994-9da9-5fecbb6e16f7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-8ed30a72-b431-4919-8471-0febebbe12ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:10.995671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:10.996546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:10.997624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 304/617] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:11.010109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed30a72-b431-4919-8471-0febebbe12ab None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:11.034834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87cf9a-6ff0-40f1-9718-e3cf609af0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:11.035526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87cf9a-6ff0-40f1-9718-e3cf609af0ef None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:11.035816 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87cf9a-6ff0-40f1-9718-e3cf609af0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:11.036083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87cf9a-6ff0-40f1-9718-e3cf609af0ef None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:11.036930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87cf9a-6ff0-40f1-9718-e3cf609af0ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:11.037290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87cf9a-6ff0-40f1-9718-e3cf609af0ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:11.038886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87cf9a-6ff0-40f1-9718-e3cf609af0ef None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:11.059890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 314/618] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jun 21 18:50:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:11.066376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87cf9a-6ff0-40f1-9718-e3cf609af0ef None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:11.091653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:11.228744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed30a72-b431-4919-8471-0febebbe12ab None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:11.263897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87cf9a-6ff0-40f1-9718-e3cf609af0ef None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:11.268126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:11.298725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed568f2-8a36-4fa8-8093-6c66999be4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:11.299403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed568f2-8a36-4fa8-8093-6c66999be4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:11.299685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed568f2-8a36-4fa8-8093-6c66999be4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:11.299967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed568f2-8a36-4fa8-8093-6c66999be4c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:11.300724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed568f2-8a36-4fa8-8093-6c66999be4c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:11.301028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed568f2-8a36-4fa8-8093-6c66999be4c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:11.302455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed568f2-8a36-4fa8-8093-6c66999be4c1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:11.311796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 305/619] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:11.318101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed568f2-8a36-4fa8-8093-6c66999be4c1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:11.334425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50878c-4dad-4fe2-b5fe-2a56f46b45aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:11.335225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50878c-4dad-4fe2-b5fe-2a56f46b45aa None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:11.335948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50878c-4dad-4fe2-b5fe-2a56f46b45aa None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:11.335948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50878c-4dad-4fe2-b5fe-2a56f46b45aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:11.352811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50878c-4dad-4fe2-b5fe-2a56f46b45aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:11.354338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50878c-4dad-4fe2-b5fe-2a56f46b45aa None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-1d50878c-4dad-4fe2-b5fe-2a56f46b45aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:11.384942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:11.386601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:11.387491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:11.387491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:11.387491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:11.387491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:11.399078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 315/620] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:11.399996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50878c-4dad-4fe2-b5fe-2a56f46b45aa None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:11.511368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:11.522272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed568f2-8a36-4fa8-8093-6c66999be4c1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:11.578296 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9f16e-5cd0-4631-a3f2-a780c213e318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:11.579003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9f16e-5cd0-4631-a3f2-a780c213e318 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:11.579285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9f16e-5cd0-4631-a3f2-a780c213e318 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:11.579558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9f16e-5cd0-4631-a3f2-a780c213e318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:11.591500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9f16e-5cd0-4631-a3f2-a780c213e318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:11.592975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9f16e-5cd0-4631-a3f2-a780c213e318 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:11.595471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50878c-4dad-4fe2-b5fe-2a56f46b45aa None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-bff9f16e-5cd0-4631-a3f2-a780c213e318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:11.611562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:11.612554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:11.613488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:11.613488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:11.613488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:11.613488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:11.617152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 306/621] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:11.618452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9f16e-5cd0-4631-a3f2-a780c213e318 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:11.661309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:11.688489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c7019-50a6-43ac-ba92-010a797fd25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:11.689214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c7019-50a6-43ac-ba92-010a797fd25d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:11.689502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c7019-50a6-43ac-ba92-010a797fd25d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:11.689780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c7019-50a6-43ac-ba92-010a797fd25d None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:11.690513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c7019-50a6-43ac-ba92-010a797fd25d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:11.690819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c7019-50a6-43ac-ba92-010a797fd25d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:11.692194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c7019-50a6-43ac-ba92-010a797fd25d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:11.712538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 316/622] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:11.718657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c7019-50a6-43ac-ba92-010a797fd25d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:11.759614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9f16e-5cd0-4631-a3f2-a780c213e318 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:11.828797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1616b-71bd-4be5-bed4-9408ffbb07b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:11.829433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1616b-71bd-4be5-bed4-9408ffbb07b0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:11.829718 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1616b-71bd-4be5-bed4-9408ffbb07b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:11.830001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1616b-71bd-4be5-bed4-9408ffbb07b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:11.830737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1616b-71bd-4be5-bed4-9408ffbb07b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:11.831053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1616b-71bd-4be5-bed4-9408ffbb07b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:11.832443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1616b-71bd-4be5-bed4-9408ffbb07b0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:11.846725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 307/623] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:11.851614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1616b-71bd-4be5-bed4-9408ffbb07b0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:11.922252 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c7019-50a6-43ac-ba92-010a797fd25d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:11.977395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:11.991580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:11.997689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1616b-71bd-4be5-bed4-9408ffbb07b0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:12.004684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f07cfc-48ed-42ce-a3da-548efebcba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:12.005578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f07cfc-48ed-42ce-a3da-548efebcba67 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.005863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f07cfc-48ed-42ce-a3da-548efebcba67 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.006141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f07cfc-48ed-42ce-a3da-548efebcba67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.020025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f07cfc-48ed-42ce-a3da-548efebcba67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:12.021514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f07cfc-48ed-42ce-a3da-548efebcba67 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-50f07cfc-48ed-42ce-a3da-548efebcba67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:12.047919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:12.048897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:12.049836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:12.049836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:12.049836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:12.049836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:12.049836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:12.049836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:12.049836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:12.049836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:12.054141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 317/624] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:12.056959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74c4f5-476a-4bc2-b1b4-f1b427dec620 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.057303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74c4f5-476a-4bc2-b1b4-f1b427dec620 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.057580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74c4f5-476a-4bc2-b1b4-f1b427dec620 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.058862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 318/625] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:12.062856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74c4f5-476a-4bc2-b1b4-f1b427dec620 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:12.067323 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc939f-47bf-4840-9228-8650ce42ce72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:12.068435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc939f-47bf-4840-9228-8650ce42ce72 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.069661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc939f-47bf-4840-9228-8650ce42ce72 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.069943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc939f-47bf-4840-9228-8650ce42ce72 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.082513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc939f-47bf-4840-9228-8650ce42ce72 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:12.084005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc939f-47bf-4840-9228-8650ce42ce72 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-08bc939f-47bf-4840-9228-8650ce42ce72 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:12.105348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:12.106377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:12.107332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 308/626] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jun 21 18:50:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:12.107979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991e0ba-f2e4-40dc-9268-5ab6bacff691 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.108613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991e0ba-f2e4-40dc-9268-5ab6bacff691 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.108613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991e0ba-f2e4-40dc-9268-5ab6bacff691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.129598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f991e0ba-f2e4-40dc-9268-5ab6bacff691 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:12.131065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f991e0ba-f2e4-40dc-9268-5ab6bacff691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:12.193568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74c4f5-476a-4bc2-b1b4-f1b427dec620 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:12.205596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 309/627] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:12.206587 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991e0ba-f2e4-40dc-9268-5ab6bacff691 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:12.259573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be78f4-e306-4656-ab9f-2fb1f6c122de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:12.260327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be78f4-e306-4656-ab9f-2fb1f6c122de None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.260613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be78f4-e306-4656-ab9f-2fb1f6c122de None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.260933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be78f4-e306-4656-ab9f-2fb1f6c122de None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.261770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be78f4-e306-4656-ab9f-2fb1f6c122de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:12.262082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be78f4-e306-4656-ab9f-2fb1f6c122de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:12.263997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be78f4-e306-4656-ab9f-2fb1f6c122de None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:12.281271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 319/628] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:12.282204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be78f4-e306-4656-ab9f-2fb1f6c122de None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:12.329490 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:12.340036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991e0ba-f2e4-40dc-9268-5ab6bacff691 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:12.345496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:12.383013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d688b9-f7d7-4a0a-9365-a6a8df23c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:12.383695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d688b9-f7d7-4a0a-9365-a6a8df23c82a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.383977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d688b9-f7d7-4a0a-9365-a6a8df23c82a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.384260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d688b9-f7d7-4a0a-9365-a6a8df23c82a None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.385284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d688b9-f7d7-4a0a-9365-a6a8df23c82a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:12.385603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d688b9-f7d7-4a0a-9365-a6a8df23c82a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:12.387271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d688b9-f7d7-4a0a-9365-a6a8df23c82a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:12.411460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 310/629] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jun 21 18:50:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:12.411460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d688b9-f7d7-4a0a-9365-a6a8df23c82a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:12.415955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be78f4-e306-4656-ab9f-2fb1f6c122de None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:12.454067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad8cbd-49ab-41e4-8d1b-8df2447aabd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:12.455339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8cbd-49ab-41e4-8d1b-8df2447aabd2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.455889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8cbd-49ab-41e4-8d1b-8df2447aabd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.456369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8cbd-49ab-41e4-8d1b-8df2447aabd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.468707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad8cbd-49ab-41e4-8d1b-8df2447aabd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-92ad8cbd-49ab-41e4-8d1b-8df2447aabd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:12.477470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:12.478392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:12.479824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:12.482939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 320/630] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:12.484024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad8cbd-49ab-41e4-8d1b-8df2447aabd2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:12.529342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d688b9-f7d7-4a0a-9365-a6a8df23c82a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:12.573589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d83e4-d7b2-4016-b707-503005bf6b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:12.575172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d83e4-d7b2-4016-b707-503005bf6b87 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.575665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d83e4-d7b2-4016-b707-503005bf6b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.576260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d83e4-d7b2-4016-b707-503005bf6b87 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/ad2a56d3235b46e3b1fbb8bf0c2b4e0b/roles/37aa79fddde643aeab12191e2c79681f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.595124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d83e4-d7b2-4016-b707-503005bf6b87 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b, role_id=37aa79fddde643aeab12191e2c79681f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:12.597196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d83e4-d7b2-4016-b707-503005bf6b87 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:12.601501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:12.614526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad8cbd-49ab-41e4-8d1b-8df2447aabd2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:12.616404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:12.617245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 311/631] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:50:12 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/ad2a56d3235b46e3b1fbb8bf0c2b4e0b/roles/37aa79fddde643aeab12191e2c79681f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:50:12.618413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d83e4-d7b2-4016-b707-503005bf6b87 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:12.668431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc04b85-1447-430c-8923-3521d9727c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:12.669463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc04b85-1447-430c-8923-3521d9727c48 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.669925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc04b85-1447-430c-8923-3521d9727c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.670378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc04b85-1447-430c-8923-3521d9727c48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.747090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d83e4-d7b2-4016-b707-503005bf6b87 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:12.753958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc04b85-1447-430c-8923-3521d9727c48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:12.756212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc04b85-1447-430c-8923-3521d9727c48 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:12.789529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395299e-7f9f-4ed1-b1cd-c00072235374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:12.790202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395299e-7f9f-4ed1-b1cd-c00072235374 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.790950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395299e-7f9f-4ed1-b1cd-c00072235374 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.790950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395299e-7f9f-4ed1-b1cd-c00072235374 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.791553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395299e-7f9f-4ed1-b1cd-c00072235374 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:12.791874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395299e-7f9f-4ed1-b1cd-c00072235374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:12.793515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395299e-7f9f-4ed1-b1cd-c00072235374 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:12.802239 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 312/632] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:12.803591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017b3b5-d347-44ac-b546-d26476d18359 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.803889 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017b3b5-d347-44ac-b546-d26476d18359 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.804178 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017b3b5-d347-44ac-b546-d26476d18359 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.805923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 313/633] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:12.808518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017b3b5-d347-44ac-b546-d26476d18359 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:12.877234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:12.892814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 321/634] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:12.892814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe96443-683b-406d-b2eb-47bd459c5b35 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.892814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe96443-683b-406d-b2eb-47bd459c5b35 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.892814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe96443-683b-406d-b2eb-47bd459c5b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.897405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:12.913116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5fe96443-683b-406d-b2eb-47bd459c5b35 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:12.914848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5fe96443-683b-406d-b2eb-47bd459c5b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:12.931303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017b3b5-d347-44ac-b546-d26476d18359 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:12.969533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d35491-d4cc-4819-94bd-35484f7707b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:12.970754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d35491-d4cc-4819-94bd-35484f7707b0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:12.971214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d35491-d4cc-4819-94bd-35484f7707b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:12.971668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d35491-d4cc-4819-94bd-35484f7707b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:12.984513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d35491-d4cc-4819-94bd-35484f7707b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:12.986350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d35491-d4cc-4819-94bd-35484f7707b0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:12.991260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 322/635] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:12.992210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe96443-683b-406d-b2eb-47bd459c5b35 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-b0d35491-d4cc-4819-94bd-35484f7707b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:13.006302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:13.007565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:13.008440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:13.008440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:13.008440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:13.008440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:13.011335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 314/636] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:13.012958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9eed44-fec7-4ced-a6d8-49496385da62 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.013442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9eed44-fec7-4ced-a6d8-49496385da62 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.013940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9eed44-fec7-4ced-a6d8-49496385da62 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.015533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 315/637] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:13.016508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9eed44-fec7-4ced-a6d8-49496385da62 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.108510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe96443-683b-406d-b2eb-47bd459c5b35 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:13.128385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9eed44-fec7-4ced-a6d8-49496385da62 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:13.140454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:13.141753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:13.148330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6d15a-f2db-4b7e-8451-62fcfa8c642f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:13.149152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6d15a-f2db-4b7e-8451-62fcfa8c642f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.149433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6d15a-f2db-4b7e-8451-62fcfa8c642f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.149704 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6d15a-f2db-4b7e-8451-62fcfa8c642f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.163394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6d15a-f2db-4b7e-8451-62fcfa8c642f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:13.164863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6d15a-f2db-4b7e-8451-62fcfa8c642f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:13.170250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e47fa-4808-423e-9891-cbaac294db32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:13.171308 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e47fa-4808-423e-9891-cbaac294db32 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.171770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e47fa-4808-423e-9891-cbaac294db32 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.172754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e47fa-4808-423e-9891-cbaac294db32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-9be6d15a-f2db-4b7e-8451-62fcfa8c642f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:13.183558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:13.184448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:13.185369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:13.185369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:13.185369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:13.185369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:13.185369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 323/638] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:13.185729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6d15a-f2db-4b7e-8451-62fcfa8c642f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.187793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e47fa-4808-423e-9891-cbaac294db32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-855e47fa-4808-423e-9891-cbaac294db32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:13.196684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:13.198451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:13.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:13.200479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 316/639] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:13.200666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef6a7a-f737-4a7f-8a20-eff1243bf7c5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.200945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef6a7a-f737-4a7f-8a20-eff1243bf7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.201253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef6a7a-f737-4a7f-8a20-eff1243bf7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.219835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d5ef6a7a-f737-4a7f-8a20-eff1243bf7c5 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:13.221042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d5ef6a7a-f737-4a7f-8a20-eff1243bf7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:13.294369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 317/640] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:13.296845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6d15a-f2db-4b7e-8451-62fcfa8c642f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:13.305778 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef6a7a-f737-4a7f-8a20-eff1243bf7c5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.343156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b220b-ab5f-4b64-a1a8-33d372f584ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:13.343818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b220b-ab5f-4b64-a1a8-33d372f584ea None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.344104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b220b-ab5f-4b64-a1a8-33d372f584ea None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.344386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b220b-ab5f-4b64-a1a8-33d372f584ea None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.345136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b220b-ab5f-4b64-a1a8-33d372f584ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:13.345443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b220b-ab5f-4b64-a1a8-33d372f584ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:13.346804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b220b-ab5f-4b64-a1a8-33d372f584ea None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:13.361291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 324/641] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:13.361291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b220b-ab5f-4b64-a1a8-33d372f584ea None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.409529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:13.424981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:13.429019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef6a7a-f737-4a7f-8a20-eff1243bf7c5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:13.466357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b220b-ab5f-4b64-a1a8-33d372f584ea None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:13.468980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8084d-61f0-4ae1-a8fc-8fad223a9242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:13.469701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8084d-61f0-4ae1-a8fc-8fad223a9242 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.469986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8084d-61f0-4ae1-a8fc-8fad223a9242 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.470287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8084d-61f0-4ae1-a8fc-8fad223a9242 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.471025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8084d-61f0-4ae1-a8fc-8fad223a9242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:13.471337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8084d-61f0-4ae1-a8fc-8fad223a9242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:13.472723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8084d-61f0-4ae1-a8fc-8fad223a9242 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:13.490290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 318/642] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:13.490290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8084d-61f0-4ae1-a8fc-8fad223a9242 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.505610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a43b7-ae93-4d83-8a07-04940e21d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:13.506286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a43b7-ae93-4d83-8a07-04940e21d73a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.506576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a43b7-ae93-4d83-8a07-04940e21d73a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.506855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a43b7-ae93-4d83-8a07-04940e21d73a None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.507604 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a43b7-ae93-4d83-8a07-04940e21d73a None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:13.507917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a43b7-ae93-4d83-8a07-04940e21d73a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:13.509321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a43b7-ae93-4d83-8a07-04940e21d73a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:13.522852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 325/643] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:13.522852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a43b7-ae93-4d83-8a07-04940e21d73a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.599677 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8084d-61f0-4ae1-a8fc-8fad223a9242 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:13.630438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a43b7-ae93-4d83-8a07-04940e21d73a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:13.636489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aca563-fb0a-46f3-ad84-e7df24525c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:13.637321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aca563-fb0a-46f3-ad84-e7df24525c29 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.637613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aca563-fb0a-46f3-ad84-e7df24525c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.637896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aca563-fb0a-46f3-ad84-e7df24525c29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.638758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:13.649461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aca563-fb0a-46f3-ad84-e7df24525c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c4aca563-fb0a-46f3-ad84-e7df24525c29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:13.657817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:13.658855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:13.659799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 319/644] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:13.661616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aca563-fb0a-46f3-ad84-e7df24525c29 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.676210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6c15c-c4cc-4829-8ec5-55e9b2866ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:13.676998 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6c15c-c4cc-4829-8ec5-55e9b2866ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.677319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6c15c-c4cc-4829-8ec5-55e9b2866ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.677597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6c15c-c4cc-4829-8ec5-55e9b2866ea9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.683999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:13.686875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6c15c-c4cc-4829-8ec5-55e9b2866ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:13.689602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6c15c-c4cc-4829-8ec5-55e9b2866ea9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-48d6c15c-c4cc-4829-8ec5-55e9b2866ea9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:13.706290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:13.707309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 326/645] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:13.708460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6c15c-c4cc-4829-8ec5-55e9b2866ea9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.773469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aca563-fb0a-46f3-ad84-e7df24525c29 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:13.812640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a720c06-bd6f-43a2-bc21-fb5aa560e70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:13.813479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a720c06-bd6f-43a2-bc21-fb5aa560e70d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.813809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a720c06-bd6f-43a2-bc21-fb5aa560e70d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.814038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a720c06-bd6f-43a2-bc21-fb5aa560e70d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.821239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6c15c-c4cc-4829-8ec5-55e9b2866ea9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:13.821741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a720c06-bd6f-43a2-bc21-fb5aa560e70d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:13.823271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a720c06-bd6f-43a2-bc21-fb5aa560e70d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-1a720c06-bd6f-43a2-bc21-fb5aa560e70d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:13.840481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:13.841524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:13.842454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:13.842454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:13.842454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:13.842454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:13.842454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:13.842454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:13.842454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:13.842454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:13.842454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 320/646] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:13.842961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a720c06-bd6f-43a2-bc21-fb5aa560e70d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.863616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b6f5d-6a55-4577-81fa-43a52b4344c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:13.864404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b6f5d-6a55-4577-81fa-43a52b4344c8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:13.864683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b6f5d-6a55-4577-81fa-43a52b4344c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:13.864968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b6f5d-6a55-4577-81fa-43a52b4344c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:13.875954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b6f5d-6a55-4577-81fa-43a52b4344c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:13.877485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b6f5d-6a55-4577-81fa-43a52b4344c8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-039b6f5d-6a55-4577-81fa-43a52b4344c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:13.895194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:13.896178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:13.897120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:13.897120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:13.897120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:13.897120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:13.897120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 327/647] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:13.897435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b6f5d-6a55-4577-81fa-43a52b4344c8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:13.918528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:13.948900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:13.971472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a720c06-bd6f-43a2-bc21-fb5aa560e70d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.003913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b6f5d-6a55-4577-81fa-43a52b4344c8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.007972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bc82a-fe75-4d88-a986-659f20646f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.008653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bc82a-fe75-4d88-a986-659f20646f31 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.008943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bc82a-fe75-4d88-a986-659f20646f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.009260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bc82a-fe75-4d88-a986-659f20646f31 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.010016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bc82a-fe75-4d88-a986-659f20646f31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:14.010373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bc82a-fe75-4d88-a986-659f20646f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.011752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bc82a-fe75-4d88-a986-659f20646f31 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.020091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 321/648] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:14.020979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bc82a-fe75-4d88-a986-659f20646f31 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:14.045778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64faa9e-bcd3-4a89-9546-8fad46dc6f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.046449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64faa9e-bcd3-4a89-9546-8fad46dc6f8c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.046731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64faa9e-bcd3-4a89-9546-8fad46dc6f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.047003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64faa9e-bcd3-4a89-9546-8fad46dc6f8c None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.047829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64faa9e-bcd3-4a89-9546-8fad46dc6f8c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:14.048135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64faa9e-bcd3-4a89-9546-8fad46dc6f8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.049583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64faa9e-bcd3-4a89-9546-8fad46dc6f8c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.066682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 328/649] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jun 21 18:50:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:14.067497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64faa9e-bcd3-4a89-9546-8fad46dc6f8c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:14.133441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bc82a-fe75-4d88-a986-659f20646f31 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.155134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:14.170654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e7000-f09b-42d7-b7a6-d8776a870447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.171305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e7000-f09b-42d7-b7a6-d8776a870447 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.171530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e7000-f09b-42d7-b7a6-d8776a870447 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.171811 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e7000-f09b-42d7-b7a6-d8776a870447 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.172608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e7000-f09b-42d7-b7a6-d8776a870447 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:14.172919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e7000-f09b-42d7-b7a6-d8776a870447 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.174442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e7000-f09b-42d7-b7a6-d8776a870447 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.182009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64faa9e-bcd3-4a89-9546-8fad46dc6f8c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.193365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 322/650] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:14.193365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e7000-f09b-42d7-b7a6-d8776a870447 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:14.193365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:14.223175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65881e-9be2-43be-a3c3-dfd7aefd01c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.223849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65881e-9be2-43be-a3c3-dfd7aefd01c5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.224136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65881e-9be2-43be-a3c3-dfd7aefd01c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.224416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65881e-9be2-43be-a3c3-dfd7aefd01c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.225180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65881e-9be2-43be-a3c3-dfd7aefd01c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:14.225496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65881e-9be2-43be-a3c3-dfd7aefd01c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.226872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65881e-9be2-43be-a3c3-dfd7aefd01c5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.242863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 329/651] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:14.242863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65881e-9be2-43be-a3c3-dfd7aefd01c5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:14.306276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e7000-f09b-42d7-b7a6-d8776a870447 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.349473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65881e-9be2-43be-a3c3-dfd7aefd01c5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.350253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69bb6d-16ca-47db-927d-7d69da555f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.351171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69bb6d-16ca-47db-927d-7d69da555f37 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.351456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69bb6d-16ca-47db-927d-7d69da555f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.351851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69bb6d-16ca-47db-927d-7d69da555f37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.363256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69bb6d-16ca-47db-927d-7d69da555f37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.364879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69bb6d-16ca-47db-927d-7d69da555f37 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-0c69bb6d-16ca-47db-927d-7d69da555f37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:14.383698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:14.384639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:14.386641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:14.386641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:14.386641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:14.386641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:14.386862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e86fc9-8552-4e2e-936f-19179e9d89a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.386862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e86fc9-8552-4e2e-936f-19179e9d89a0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.387058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 323/652] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:14.387058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69bb6d-16ca-47db-927d-7d69da555f37 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:14.390005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e86fc9-8552-4e2e-936f-19179e9d89a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.390338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e86fc9-8552-4e2e-936f-19179e9d89a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.394959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:14.449265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:14.467689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e86fc9-8552-4e2e-936f-19179e9d89a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.468889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e86fc9-8552-4e2e-936f-19179e9d89a0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.499398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69bb6d-16ca-47db-927d-7d69da555f37 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.538965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe00118-3e86-4178-b337-537a95f00f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.539967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe00118-3e86-4178-b337-537a95f00f79 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.540452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe00118-3e86-4178-b337-537a95f00f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.540742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe00118-3e86-4178-b337-537a95f00f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.542157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe00118-3e86-4178-b337-537a95f00f79 None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=dcbcec65220c4ebb95f974925fa73f34) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:14.542481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe00118-3e86-4178-b337-537a95f00f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.543939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe00118-3e86-4178-b337-537a95f00f79 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.550426 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 324/653] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/role_assignments?user.id=dcbcec65220c4ebb95f974925fa73f34&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:14.551263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe00118-3e86-4178-b337-537a95f00f79 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:14.598305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 330/654] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:14.599685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e86fc9-8552-4e2e-936f-19179e9d89a0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:14.647986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:14.709147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe00118-3e86-4178-b337-537a95f00f79 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.733901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:14.736107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e86fc9-8552-4e2e-936f-19179e9d89a0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.760833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb5a1-23ad-46ee-b385-320cebf3edf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.762708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb5a1-23ad-46ee-b385-320cebf3edf2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.762708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb5a1-23ad-46ee-b385-320cebf3edf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.762708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956eb5a1-23ad-46ee-b385-320cebf3edf2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.774918 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956eb5a1-23ad-46ee-b385-320cebf3edf2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.776932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956eb5a1-23ad-46ee-b385-320cebf3edf2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.778574 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76176ce-3a1f-40e6-8850-53beef7ec321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.779686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76176ce-3a1f-40e6-8850-53beef7ec321 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.779686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76176ce-3a1f-40e6-8850-53beef7ec321 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.779686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76176ce-3a1f-40e6-8850-53beef7ec321 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.781016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76176ce-3a1f-40e6-8850-53beef7ec321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:14.781296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76176ce-3a1f-40e6-8850-53beef7ec321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.782668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76176ce-3a1f-40e6-8850-53beef7ec321 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-956eb5a1-23ad-46ee-b385-320cebf3edf2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:14.793966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:14.794927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:14.796233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 325/655] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:14.797030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 331/656] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:14.797138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb5a1-23ad-46ee-b385-320cebf3edf2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:14.797224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3165bf-89e7-4935-aa49-3b887fa07f43 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.797224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3165bf-89e7-4935-aa49-3b887fa07f43 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.797356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3165bf-89e7-4935-aa49-3b887fa07f43 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.798678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 332/657] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:14.803864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3165bf-89e7-4935-aa49-3b887fa07f43 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:14.913529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956eb5a1-23ad-46ee-b385-320cebf3edf2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.913958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:14.922038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3165bf-89e7-4935-aa49-3b887fa07f43 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:14.952095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d3209-cd08-4027-9527-fbcf5558b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.952895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d3209-cd08-4027-9527-fbcf5558b34b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.953213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d3209-cd08-4027-9527-fbcf5558b34b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.953506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d3209-cd08-4027-9527-fbcf5558b34b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.957381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745e329-96b5-4750-a3f7-9b85f159f1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:14.958172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745e329-96b5-4750-a3f7-9b85f159f1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.958453 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745e329-96b5-4750-a3f7-9b85f159f1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.958727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745e329-96b5-4750-a3f7-9b85f159f1e5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.965324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d3209-cd08-4027-9527-fbcf5558b34b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.966731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d3209-cd08-4027-9527-fbcf5558b34b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.967164 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745e329-96b5-4750-a3f7-9b85f159f1e5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:14.968581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745e329-96b5-4750-a3f7-9b85f159f1e5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d745e329-96b5-4750-a3f7-9b85f159f1e5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:14.983157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:14.984171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 18:50:14.985134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-585d3209-cd08-4027-9527-fbcf5558b34b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:14.990760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:14.991632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:14.992522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:14.992522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:14.992522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:14.992522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:14.992522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 326/658] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:14.992522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402720f-109e-4af2-a24b-8f855f26b619 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:14.992522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402720f-109e-4af2-a24b-8f855f26b619 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:14.992522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402720f-109e-4af2-a24b-8f855f26b619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:14.992995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 333/659] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:14.992995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745e329-96b5-4750-a3f7-9b85f159f1e5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:15.009839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e402720f-109e-4af2-a24b-8f855f26b619 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:15.011649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e402720f-109e-4af2-a24b-8f855f26b619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:15.042241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:15.093842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 327/660] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:15.094983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402720f-109e-4af2-a24b-8f855f26b619 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:15.103839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745e329-96b5-4750-a3f7-9b85f159f1e5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:15.144504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55190e73-d642-47fe-9a30-a68f4a244b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:15.145199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55190e73-d642-47fe-9a30-a68f4a244b93 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:15.145481 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55190e73-d642-47fe-9a30-a68f4a244b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:15.145783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55190e73-d642-47fe-9a30-a68f4a244b93 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:15.146588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55190e73-d642-47fe-9a30-a68f4a244b93 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:15.146897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55190e73-d642-47fe-9a30-a68f4a244b93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:15.148311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55190e73-d642-47fe-9a30-a68f4a244b93 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:15.168521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 334/661] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jun 21 18:50:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:15.169368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55190e73-d642-47fe-9a30-a68f4a244b93 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:15.186090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:15.209242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402720f-109e-4af2-a24b-8f855f26b619 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:15.246629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83bae5-3da5-42c5-abe6-764c176c57fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:15.247216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83bae5-3da5-42c5-abe6-764c176c57fd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:15.247497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83bae5-3da5-42c5-abe6-764c176c57fd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:15.247773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83bae5-3da5-42c5-abe6-764c176c57fd None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:15.248633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83bae5-3da5-42c5-abe6-764c176c57fd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:15.248943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83bae5-3da5-42c5-abe6-764c176c57fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:15.250407 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83bae5-3da5-42c5-abe6-764c176c57fd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:15.268627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 328/662] 10.222.0.70 () {56 vars in 1139 bytes} [Tue Jun 21 18:50:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:15.269486 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83bae5-3da5-42c5-abe6-764c176c57fd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:15.279042 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55190e73-d642-47fe-9a30-a68f4a244b93 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:15.296610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:15.316756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c932ac-877a-456c-af38-e4b453720cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:15.317832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c932ac-877a-456c-af38-e4b453720cab None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:15.318116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c932ac-877a-456c-af38-e4b453720cab None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:15.318387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c932ac-877a-456c-af38-e4b453720cab None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:15.319112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c932ac-877a-456c-af38-e4b453720cab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:15.319418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c932ac-877a-456c-af38-e4b453720cab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:15.320791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c932ac-877a-456c-af38-e4b453720cab None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:15.337138 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 335/663] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:15.338323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c932ac-877a-456c-af38-e4b453720cab None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:15.397620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83bae5-3da5-42c5-abe6-764c176c57fd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:15.420123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:15.438004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67f4dc-a2af-4d1c-8918-924232b0d172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:15.439071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67f4dc-a2af-4d1c-8918-924232b0d172 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:15.439510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67f4dc-a2af-4d1c-8918-924232b0d172 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:15.439928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67f4dc-a2af-4d1c-8918-924232b0d172 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:15.448959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c932ac-877a-456c-af38-e4b453720cab None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:15.451961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67f4dc-a2af-4d1c-8918-924232b0d172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2d67f4dc-a2af-4d1c-8918-924232b0d172 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:15.459646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:15.460618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 329/664] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:15.461846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67f4dc-a2af-4d1c-8918-924232b0d172 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:15.488992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495e54f-9556-47ea-95e2-090080c4fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:15.489908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495e54f-9556-47ea-95e2-090080c4fb04 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:15.490365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495e54f-9556-47ea-95e2-090080c4fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:15.490820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495e54f-9556-47ea-95e2-090080c4fb04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:15.547732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:15.564384 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495e54f-9556-47ea-95e2-090080c4fb04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:15.567705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495e54f-9556-47ea-95e2-090080c4fb04 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:15.574934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67f4dc-a2af-4d1c-8918-924232b0d172 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:15.609276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411421be-5591-41d0-b333-e3b1658d3fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:15.609939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411421be-5591-41d0-b333-e3b1658d3fdb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:15.610211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411421be-5591-41d0-b333-e3b1658d3fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:15.610494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411421be-5591-41d0-b333-e3b1658d3fdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:15.648214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:15.678811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 336/665] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:15.679669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495e54f-9556-47ea-95e2-090080c4fb04 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:15.688463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411421be-5591-41d0-b333-e3b1658d3fdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:15.690174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411421be-5591-41d0-b333-e3b1658d3fdb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:15.771867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:15.793922 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495e54f-9556-47ea-95e2-090080c4fb04 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:15.803372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 330/666] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:15.804367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411421be-5591-41d0-b333-e3b1658d3fdb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:15.836976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dc12c-aa5a-48d8-9fee-a2d9f43e40de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:15.837875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc12c-aa5a-48d8-9fee-a2d9f43e40de None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:15.838200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc12c-aa5a-48d8-9fee-a2d9f43e40de None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:15.838518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc12c-aa5a-48d8-9fee-a2d9f43e40de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:15.845828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dc12c-aa5a-48d8-9fee-a2d9f43e40de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:15.847297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dc12c-aa5a-48d8-9fee-a2d9f43e40de None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-948dc12c-aa5a-48d8-9fee-a2d9f43e40de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:15.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:15.863154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:15.864034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:15.864034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:15.864034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:15.864034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:15.864034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:15.864034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:15.864034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:15.864034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:15.864034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 337/667] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:15.864532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dc12c-aa5a-48d8-9fee-a2d9f43e40de None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:15.907078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:15.933680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411421be-5591-41d0-b333-e3b1658d3fdb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:15.973621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e9942-672c-4241-b694-6a060ac039bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:15.974933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9942-672c-4241-b694-6a060ac039bf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:15.975684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9942-672c-4241-b694-6a060ac039bf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:15.976184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9942-672c-4241-b694-6a060ac039bf None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:15.977758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e9942-672c-4241-b694-6a060ac039bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:15.978250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e9942-672c-4241-b694-6a060ac039bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:15.979875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dc12c-aa5a-48d8-9fee-a2d9f43e40de None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:15.980592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e9942-672c-4241-b694-6a060ac039bf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:15.992866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 331/668] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:15.994566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e9942-672c-4241-b694-6a060ac039bf None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.015085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015938a-532e-48d7-87b8-080a227a06f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.015892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015938a-532e-48d7-87b8-080a227a06f1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.016180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015938a-532e-48d7-87b8-080a227a06f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.016461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015938a-532e-48d7-87b8-080a227a06f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.027580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015938a-532e-48d7-87b8-080a227a06f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.029037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015938a-532e-48d7-87b8-080a227a06f1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3015938a-532e-48d7-87b8-080a227a06f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:16.048425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:16.049596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:16.050510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:16.050510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:16.050510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:16.050510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:16.050510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 338/669] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:16.050793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015938a-532e-48d7-87b8-080a227a06f1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.050951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:16.120754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e9942-672c-4241-b694-6a060ac039bf None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.143628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:16.158701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86075b8-3cf8-4937-8f46-cc83e027fa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.159504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86075b8-3cf8-4937-8f46-cc83e027fa86 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.159793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86075b8-3cf8-4937-8f46-cc83e027fa86 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.160071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86075b8-3cf8-4937-8f46-cc83e027fa86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.160414 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015938a-532e-48d7-87b8-080a227a06f1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.172254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86075b8-3cf8-4937-8f46-cc83e027fa86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.173747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86075b8-3cf8-4937-8f46-cc83e027fa86 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a86075b8-3cf8-4937-8f46-cc83e027fa86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:16.192458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:16.193625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:16.194700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:16.194700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:16.194700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:16.194700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:16.198479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 332/670] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:16.199529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86075b8-3cf8-4937-8f46-cc83e027fa86 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.202062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff273-8d93-43bd-be5e-8224973aa739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.202726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff273-8d93-43bd-be5e-8224973aa739 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.203007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff273-8d93-43bd-be5e-8224973aa739 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.203334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff273-8d93-43bd-be5e-8224973aa739 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.204066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eff273-8d93-43bd-be5e-8224973aa739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:16.204377 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eff273-8d93-43bd-be5e-8224973aa739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.205869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eff273-8d93-43bd-be5e-8224973aa739 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.220473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 339/671] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:16.221349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff273-8d93-43bd-be5e-8224973aa739 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.308850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:16.324094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86075b8-3cf8-4937-8f46-cc83e027fa86 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.342305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff273-8d93-43bd-be5e-8224973aa739 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.366620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1f1e8-c9f1-4d39-8726-6d6182167f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.367448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f1e8-c9f1-4d39-8726-6d6182167f51 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.368292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f1e8-c9f1-4d39-8726-6d6182167f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.368623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f1e8-c9f1-4d39-8726-6d6182167f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.381596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1f1e8-c9f1-4d39-8726-6d6182167f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.383006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1f1e8-c9f1-4d39-8726-6d6182167f51 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.383142 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1c576-0197-4fb4-899d-0d55d3582aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.383867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1c576-0197-4fb4-899d-0d55d3582aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.384250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1c576-0197-4fb4-899d-0d55d3582aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.384592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1c576-0197-4fb4-899d-0d55d3582aa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.385538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1c576-0197-4fb4-899d-0d55d3582aa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:16.385919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1c576-0197-4fb4-899d-0d55d3582aa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.387401 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1c576-0197-4fb4-899d-0d55d3582aa9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.406304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 340/672] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:16.406304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1c576-0197-4fb4-899d-0d55d3582aa9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-46c1f1e8-c9f1-4d39-8726-6d6182167f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:16.406632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:16.407443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:16.408336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:16.408336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:16.408336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:16.408336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:16.408336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 333/673] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:16.408336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1f1e8-c9f1-4d39-8726-6d6182167f51 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.420177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:16.523879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1f1e8-c9f1-4d39-8726-6d6182167f51 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.524285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1c576-0197-4fb4-899d-0d55d3582aa9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.560016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41cd33-8a5d-4280-82f5-4fb135f88eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.560680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41cd33-8a5d-4280-82f5-4fb135f88eee None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.560969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41cd33-8a5d-4280-82f5-4fb135f88eee None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.561279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41cd33-8a5d-4280-82f5-4fb135f88eee None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.561567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2bbf1-9919-4c26-8e3b-21617ab5712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.562008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41cd33-8a5d-4280-82f5-4fb135f88eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:16.562469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2bbf1-9919-4c26-8e3b-21617ab5712b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.562579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41cd33-8a5d-4280-82f5-4fb135f88eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.562760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2bbf1-9919-4c26-8e3b-21617ab5712b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.563035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2bbf1-9919-4c26-8e3b-21617ab5712b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.563856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2bbf1-9919-4c26-8e3b-21617ab5712b None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=915704f26c814eaa9e6c2a68533a9c66) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:16.563951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41cd33-8a5d-4280-82f5-4fb135f88eee None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.564164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2bbf1-9919-4c26-8e3b-21617ab5712b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.565747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2bbf1-9919-4c26-8e3b-21617ab5712b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.571951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 334/674] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/role_assignments?user.id=915704f26c814eaa9e6c2a68533a9c66&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:16.573317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2bbf1-9919-4c26-8e3b-21617ab5712b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.585159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:16.590398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 341/675] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:16.591681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41cd33-8a5d-4280-82f5-4fb135f88eee None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.683849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:16.708340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2bbf1-9919-4c26-8e3b-21617ab5712b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.733689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41cd33-8a5d-4280-82f5-4fb135f88eee None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.750266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec284473-f236-44a8-9d58-b914aa13d8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.750914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec284473-f236-44a8-9d58-b914aa13d8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.751197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec284473-f236-44a8-9d58-b914aa13d8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.751475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec284473-f236-44a8-9d58-b914aa13d8d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.752211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec284473-f236-44a8-9d58-b914aa13d8d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:16.752630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec284473-f236-44a8-9d58-b914aa13d8d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.754007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec284473-f236-44a8-9d58-b914aa13d8d9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.767814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 335/676] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:16.768661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec284473-f236-44a8-9d58-b914aa13d8d9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.774694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a1068-2090-476f-a59e-3ffb06167516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.775495 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a1068-2090-476f-a59e-3ffb06167516 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.775867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a1068-2090-476f-a59e-3ffb06167516 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.776675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a1068-2090-476f-a59e-3ffb06167516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.784151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a1068-2090-476f-a59e-3ffb06167516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.785678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a1068-2090-476f-a59e-3ffb06167516 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-926a1068-2090-476f-a59e-3ffb06167516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:16.800253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:16.801328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:16.802300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:16.802300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:16.802300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:16.802300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:16.802300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:16.802300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:16.802300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:16.802300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:16.805255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 342/677] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:16.806775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec373196-fba9-4542-b47b-b76ec0882e3f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.807753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec373196-fba9-4542-b47b-b76ec0882e3f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.808017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec373196-fba9-4542-b47b-b76ec0882e3f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.809586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 343/678] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:16.810152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec373196-fba9-4542-b47b-b76ec0882e3f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.857205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:16.895762 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec284473-f236-44a8-9d58-b914aa13d8d9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.915369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec373196-fba9-4542-b47b-b76ec0882e3f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:16.933652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0150069-d047-49cf-ad00-5a0768a2f532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.934503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0150069-d047-49cf-ad00-5a0768a2f532 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.934736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0150069-d047-49cf-ad00-5a0768a2f532 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.935018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0150069-d047-49cf-ad00-5a0768a2f532 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.941541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:16.944539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0150069-d047-49cf-ad00-5a0768a2f532 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.946637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0150069-d047-49cf-ad00-5a0768a2f532 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.951150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926d988-b429-47f2-9885-d493a5cbde38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:16.951944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d988-b429-47f2-9885-d493a5cbde38 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.952225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d988-b429-47f2-9885-d493a5cbde38 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.952499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d988-b429-47f2-9885-d493a5cbde38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:16.961236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926d988-b429-47f2-9885-d493a5cbde38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a0150069-d047-49cf-ad00-5a0768a2f532 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:16.962475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:16.963379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:16.964305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:16.964305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:16.964305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:16.964305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:16.964305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:16.964305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:16.964305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:16.964305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:16.964736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926d988-b429-47f2-9885-d493a5cbde38 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:16.964808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 336/679] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:16.964876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0150069-d047-49cf-ad00-5a0768a2f532 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a926d988-b429-47f2-9885-d493a5cbde38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:16.980557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:16.981699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:16.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 344/680] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:16.983339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28d2a5-b529-482c-8ab9-b3dbc3e704cc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:16.983339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28d2a5-b529-482c-8ab9-b3dbc3e704cc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:16.983836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28d2a5-b529-482c-8ab9-b3dbc3e704cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:17.001824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6c28d2a5-b529-482c-8ab9-b3dbc3e704cc None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:17.003035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6c28d2a5-b529-482c-8ab9-b3dbc3e704cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:17.077049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0150069-d047-49cf-ad00-5a0768a2f532 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:17.085140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 345/681] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:17.085140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28d2a5-b529-482c-8ab9-b3dbc3e704cc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:17.107563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:17.117527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4bd3c4-74e4-4927-8f24-da1cd0eaea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:17.118201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bd3c4-74e4-4927-8f24-da1cd0eaea98 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:17.118488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bd3c4-74e4-4927-8f24-da1cd0eaea98 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:17.118769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bd3c4-74e4-4927-8f24-da1cd0eaea98 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:17.120108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bd3c4-74e4-4927-8f24-da1cd0eaea98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:17.120423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bd3c4-74e4-4927-8f24-da1cd0eaea98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:17.121901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bd3c4-74e4-4927-8f24-da1cd0eaea98 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:17.140635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 337/682] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:17.140635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4bd3c4-74e4-4927-8f24-da1cd0eaea98 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:17.198162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28d2a5-b529-482c-8ab9-b3dbc3e704cc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:17.215471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:17.233898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a39fdf-7480-4f78-9342-d633b3f051bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:17.234561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a39fdf-7480-4f78-9342-d633b3f051bf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:17.234841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a39fdf-7480-4f78-9342-d633b3f051bf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:17.235121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a39fdf-7480-4f78-9342-d633b3f051bf None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:17.235925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a39fdf-7480-4f78-9342-d633b3f051bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:17.236221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a39fdf-7480-4f78-9342-d633b3f051bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:17.237687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a39fdf-7480-4f78-9342-d633b3f051bf None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:17.246543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4bd3c4-74e4-4927-8f24-da1cd0eaea98 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:17.255063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 346/683] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:17.255730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a39fdf-7480-4f78-9342-d633b3f051bf None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:17.282395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed792f2b-6039-4b74-a582-6bdbc591e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:17.283077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed792f2b-6039-4b74-a582-6bdbc591e560 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:17.283370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed792f2b-6039-4b74-a582-6bdbc591e560 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:17.283651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed792f2b-6039-4b74-a582-6bdbc591e560 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:17.284508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed792f2b-6039-4b74-a582-6bdbc591e560 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:17.284822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed792f2b-6039-4b74-a582-6bdbc591e560 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:17.286311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed792f2b-6039-4b74-a582-6bdbc591e560 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:17.305362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 338/684] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jun 21 18:50:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:17.306223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed792f2b-6039-4b74-a582-6bdbc591e560 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:17.365652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:17.380557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a39fdf-7480-4f78-9342-d633b3f051bf None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:17.422319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93063d-e5ab-4a6d-83e5-d5b1de456db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:17.423116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93063d-e5ab-4a6d-83e5-d5b1de456db9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:17.423406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93063d-e5ab-4a6d-83e5-d5b1de456db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:17.423687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93063d-e5ab-4a6d-83e5-d5b1de456db9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:17.424723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed792f2b-6039-4b74-a582-6bdbc591e560 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:17.435520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93063d-e5ab-4a6d-83e5-d5b1de456db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-1d93063d-e5ab-4a6d-83e5-d5b1de456db9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:17.443485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:17.444427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 347/685] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:17.445707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93063d-e5ab-4a6d-83e5-d5b1de456db9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:17.470519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878a0f9-96c9-4838-bb57-efc3978580ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:17.471101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878a0f9-96c9-4838-bb57-efc3978580ae None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:17.471391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878a0f9-96c9-4838-bb57-efc3978580ae None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:17.471678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878a0f9-96c9-4838-bb57-efc3978580ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:17.472517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878a0f9-96c9-4838-bb57-efc3978580ae None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=5b8161446e1344108f8283d46da23ab9) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:17.472838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878a0f9-96c9-4838-bb57-efc3978580ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:17.474355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878a0f9-96c9-4838-bb57-efc3978580ae None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:17.481500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 339/686] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:17 2022] GET /identity/v3/role_assignments?user.id=5b8161446e1344108f8283d46da23ab9&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:17.482226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878a0f9-96c9-4838-bb57-efc3978580ae None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:17.488810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:17.571754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93063d-e5ab-4a6d-83e5-d5b1de456db9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:17.623973 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e27338-17f2-472a-aa0e-bc4924380fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:17.625130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e27338-17f2-472a-aa0e-bc4924380fda None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:17.625882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e27338-17f2-472a-aa0e-bc4924380fda None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:17.626359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e27338-17f2-472a-aa0e-bc4924380fda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:17.628175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e27338-17f2-472a-aa0e-bc4924380fda None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=808b530d24b3487b9a20ee74c131502e) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:17.628742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e27338-17f2-472a-aa0e-bc4924380fda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:17.631509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e27338-17f2-472a-aa0e-bc4924380fda None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:17.633351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878a0f9-96c9-4838-bb57-efc3978580ae None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:17.645187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 348/687] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:17 2022] GET /identity/v3/role_assignments?user.id=808b530d24b3487b9a20ee74c131502e&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:17.645187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e27338-17f2-472a-aa0e-bc4924380fda None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:17.651612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:17.692899 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46397abf-de10-4ae2-8403-2a438b64e5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:17.693827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46397abf-de10-4ae2-8403-2a438b64e5ae None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:17.694123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46397abf-de10-4ae2-8403-2a438b64e5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:17.694408 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46397abf-de10-4ae2-8403-2a438b64e5ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:17.772452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:17.775495 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e27338-17f2-472a-aa0e-bc4924380fda None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:17.784937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46397abf-de10-4ae2-8403-2a438b64e5ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:17.787053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46397abf-de10-4ae2-8403-2a438b64e5ae None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:17.817260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db360238-0b4d-4f9d-9872-7cf6c1043dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:17.817948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db360238-0b4d-4f9d-9872-7cf6c1043dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:17.818240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db360238-0b4d-4f9d-9872-7cf6c1043dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:17.818524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db360238-0b4d-4f9d-9872-7cf6c1043dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:17.819294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db360238-0b4d-4f9d-9872-7cf6c1043dc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:17.819607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db360238-0b4d-4f9d-9872-7cf6c1043dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:17.821257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db360238-0b4d-4f9d-9872-7cf6c1043dc7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:17.831586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 349/688] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:17.836686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db360238-0b4d-4f9d-9872-7cf6c1043dc7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:17.923537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 340/689] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:17.930756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46397abf-de10-4ae2-8403-2a438b64e5ae None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:17.942782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:17.990641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db360238-0b4d-4f9d-9872-7cf6c1043dc7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:18.036494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fbfed-0601-49b6-82a5-54d1dfbb4914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:18.037271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fbfed-0601-49b6-82a5-54d1dfbb4914 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.037560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fbfed-0601-49b6-82a5-54d1dfbb4914 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.037939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fbfed-0601-49b6-82a5-54d1dfbb4914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.050797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fbfed-0601-49b6-82a5-54d1dfbb4914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:18.052361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fbfed-0601-49b6-82a5-54d1dfbb4914 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:18.062538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46397abf-de10-4ae2-8403-2a438b64e5ae None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a48fbfed-0601-49b6-82a5-54d1dfbb4914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:18.073101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:18.074053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:18.074939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:18.074939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:18.074939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:18.074939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:18.077191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 350/690] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:18.081342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fbfed-0601-49b6-82a5-54d1dfbb4914 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:18.101335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:18.108632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469b380-07de-465b-9404-bb053e4c8ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:18.109482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469b380-07de-465b-9404-bb053e4c8ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.109774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469b380-07de-465b-9404-bb053e4c8ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.110062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469b380-07de-465b-9404-bb053e4c8ce9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.125803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469b380-07de-465b-9404-bb053e4c8ce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:18.128277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469b380-07de-465b-9404-bb053e4c8ce9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a469b380-07de-465b-9404-bb053e4c8ce9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:18.153560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:18.154710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:18.156075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:18.156075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:18.156075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:18.156075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:18.156075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 341/691] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:18.160877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469b380-07de-465b-9404-bb053e4c8ce9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:18.206071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fbfed-0601-49b6-82a5-54d1dfbb4914 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:18.244206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239a557-f742-420c-8c65-bd149adc07b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:18.245647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239a557-f742-420c-8c65-bd149adc07b1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.245932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239a557-f742-420c-8c65-bd149adc07b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.246248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239a557-f742-420c-8c65-bd149adc07b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.247099 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239a557-f742-420c-8c65-bd149adc07b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:18.247496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239a557-f742-420c-8c65-bd149adc07b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:18.249022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239a557-f742-420c-8c65-bd149adc07b1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:18.260954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:18.264118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 351/692] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:18.269548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239a557-f742-420c-8c65-bd149adc07b1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:18.314285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469b380-07de-465b-9404-bb053e4c8ce9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:18.353934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da339a9f-299f-444d-93f5-527f269d5512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:18.355124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339a9f-299f-444d-93f5-527f269d5512 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.355413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339a9f-299f-444d-93f5-527f269d5512 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.355698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339a9f-299f-444d-93f5-527f269d5512 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.356438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da339a9f-299f-444d-93f5-527f269d5512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:18.356752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da339a9f-299f-444d-93f5-527f269d5512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:18.361740 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da339a9f-299f-444d-93f5-527f269d5512 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:18.384813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 342/693] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:18.391731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da339a9f-299f-444d-93f5-527f269d5512 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:18.415544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:18.469782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239a557-f742-420c-8c65-bd149adc07b1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:18.507687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1171ecb-46ce-445b-b868-b15f1595971b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:18.508482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1171ecb-46ce-445b-b868-b15f1595971b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.508765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1171ecb-46ce-445b-b868-b15f1595971b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.509041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1171ecb-46ce-445b-b868-b15f1595971b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.517428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1171ecb-46ce-445b-b868-b15f1595971b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:18.518883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1171ecb-46ce-445b-b868-b15f1595971b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d1171ecb-46ce-445b-b868-b15f1595971b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:18.535205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:18.536217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:18.537198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 352/694] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jun 21 18:50:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:18.542826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1171ecb-46ce-445b-b868-b15f1595971b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:18.615795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da339a9f-299f-444d-93f5-527f269d5512 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:18.654092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1171ecb-46ce-445b-b868-b15f1595971b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:18.664064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:18.692891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506ab7b-0f31-4cde-8fe0-4e1a2cb012a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:18.693726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506ab7b-0f31-4cde-8fe0-4e1a2cb012a4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.694010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506ab7b-0f31-4cde-8fe0-4e1a2cb012a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.694291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506ab7b-0f31-4cde-8fe0-4e1a2cb012a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.697472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd6f79-960e-4a4f-910b-dcf84ea3259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:18.698729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd6f79-960e-4a4f-910b-dcf84ea3259b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.699016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd6f79-960e-4a4f-910b-dcf84ea3259b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.699287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd6f79-960e-4a4f-910b-dcf84ea3259b None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.700124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd6f79-960e-4a4f-910b-dcf84ea3259b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:18.700763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd6f79-960e-4a4f-910b-dcf84ea3259b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:18.702220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd6f79-960e-4a4f-910b-dcf84ea3259b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:18.704735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:18.716257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506ab7b-0f31-4cde-8fe0-4e1a2cb012a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:18.717782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506ab7b-0f31-4cde-8fe0-4e1a2cb012a4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:18.742364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 353/695] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jun 21 18:50:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-e506ab7b-0f31-4cde-8fe0-4e1a2cb012a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:18.749895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:18.750823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:18.751710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:18.751710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:18.751710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:18.751710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:18.751710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:18.751710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:18.751710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:18.751710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:18.751710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 343/696] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:18.753556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd6f79-960e-4a4f-910b-dcf84ea3259b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:18.758673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506ab7b-0f31-4cde-8fe0-4e1a2cb012a4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:18.867165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd6f79-960e-4a4f-910b-dcf84ea3259b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:18.903337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506ab7b-0f31-4cde-8fe0-4e1a2cb012a4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:18.912679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e1a83-54b2-4480-baa1-5b9bd5af2895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:18.913519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e1a83-54b2-4480-baa1-5b9bd5af2895 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.914373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e1a83-54b2-4480-baa1-5b9bd5af2895 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.914659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e1a83-54b2-4480-baa1-5b9bd5af2895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.941061 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892dc8c-5831-49a4-a5eb-fb92f62fe5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:18.941771 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892dc8c-5831-49a4-a5eb-fb92f62fe5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.942060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892dc8c-5831-49a4-a5eb-fb92f62fe5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.942344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892dc8c-5831-49a4-a5eb-fb92f62fe5e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.943148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892dc8c-5831-49a4-a5eb-fb92f62fe5e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:18.943460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892dc8c-5831-49a4-a5eb-fb92f62fe5e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:18.944911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892dc8c-5831-49a4-a5eb-fb92f62fe5e2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:18.958728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 344/697] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:18.960175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673f339-7254-4139-9b39-6fee5a34a2dd None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:18.960482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673f339-7254-4139-9b39-6fee5a34a2dd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:18.960876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673f339-7254-4139-9b39-6fee5a34a2dd None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:18.962586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 345/698] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:18.965828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673f339-7254-4139-9b39-6fee5a34a2dd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:18.996936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:19.009852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e1a83-54b2-4480-baa1-5b9bd5af2895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:19.016614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e1a83-54b2-4480-baa1-5b9bd5af2895 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:19.021035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:19.092659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673f339-7254-4139-9b39-6fee5a34a2dd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:19.131026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527179f-5c63-4511-8d0e-18c39906b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:19.132466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527179f-5c63-4511-8d0e-18c39906b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:19.132960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527179f-5c63-4511-8d0e-18c39906b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:19.133467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527179f-5c63-4511-8d0e-18c39906b2b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:19.134561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527179f-5c63-4511-8d0e-18c39906b2b1 None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:19.135676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527179f-5c63-4511-8d0e-18c39906b2b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:19.137417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527179f-5c63-4511-8d0e-18c39906b2b1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:19.139782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 354/699] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:19.141556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd59a-d539-415a-85cc-4235171de079 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:19.141849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd59a-d539-415a-85cc-4235171de079 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:19.142130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd59a-d539-415a-85cc-4235171de079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:19.147211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 346/700] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:18 2022] GET /identity/v3/role_assignments?user.id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:19.148517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527179f-5c63-4511-8d0e-18c39906b2b1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:19.160472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9cabd59a-d539-415a-85cc-4235171de079 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:19.161764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9cabd59a-d539-415a-85cc-4235171de079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:19.247716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 355/701] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:19.268682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527179f-5c63-4511-8d0e-18c39906b2b1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:19.277444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:19.279641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd59a-d539-415a-85cc-4235171de079 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:19.310273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56966587-436b-4a92-aa20-675dc2516299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:19.311092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56966587-436b-4a92-aa20-675dc2516299 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:19.311384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56966587-436b-4a92-aa20-675dc2516299 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:19.311672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56966587-436b-4a92-aa20-675dc2516299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:19.340906 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:19.343740 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56966587-436b-4a92-aa20-675dc2516299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:19.346794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56966587-436b-4a92-aa20-675dc2516299 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-56966587-436b-4a92-aa20-675dc2516299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:19.372492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:19.373654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:19.374858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:19.374858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:19.374858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:19.374858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:19.376617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 347/702] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:19.386569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56966587-436b-4a92-aa20-675dc2516299 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:19.461668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd59a-d539-415a-85cc-4235171de079 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:19.499616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee6112-e65e-4820-bbae-f5ecdc9110e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:19.500659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee6112-e65e-4820-bbae-f5ecdc9110e3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:19.501157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee6112-e65e-4820-bbae-f5ecdc9110e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:19.501641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee6112-e65e-4820-bbae-f5ecdc9110e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:19.513539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee6112-e65e-4820-bbae-f5ecdc9110e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:19.515286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee6112-e65e-4820-bbae-f5ecdc9110e3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:19.524264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56966587-436b-4a92-aa20-675dc2516299 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-0fee6112-e65e-4820-bbae-f5ecdc9110e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:19.535499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:19.536423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:19.537368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:19.537368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:19.537368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:19.537368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:19.539701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 356/703] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:19.545454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee6112-e65e-4820-bbae-f5ecdc9110e3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:19.567203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74aadb3-d59f-494b-8309-c6453b0e27da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:19.567875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74aadb3-d59f-494b-8309-c6453b0e27da None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:19.568164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74aadb3-d59f-494b-8309-c6453b0e27da None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:19.568446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74aadb3-d59f-494b-8309-c6453b0e27da None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:19.569300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74aadb3-d59f-494b-8309-c6453b0e27da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:19.569645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74aadb3-d59f-494b-8309-c6453b0e27da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:19.571047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74aadb3-d59f-494b-8309-c6453b0e27da None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:19.584427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 348/704] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:19.594151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74aadb3-d59f-494b-8309-c6453b0e27da None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:19.617831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:19.638571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:19.666305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee6112-e65e-4820-bbae-f5ecdc9110e3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:19.704561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916a549-4b64-4a78-be46-743a053e0cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:19.706339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916a549-4b64-4a78-be46-743a053e0cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:19.706803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916a549-4b64-4a78-be46-743a053e0cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:19.707228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916a549-4b64-4a78-be46-743a053e0cb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:19.708128 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916a549-4b64-4a78-be46-743a053e0cb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:19.708587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916a549-4b64-4a78-be46-743a053e0cb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:19.710844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916a549-4b64-4a78-be46-743a053e0cb9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:19.723758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74aadb3-d59f-494b-8309-c6453b0e27da None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:19.724228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 357/705] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:19.729122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916a549-4b64-4a78-be46-743a053e0cb9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:19.768350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ab5b-508c-49dd-9458-b2f98af24bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:19.769196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ab5b-508c-49dd-9458-b2f98af24bdc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:19.769486 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ab5b-508c-49dd-9458-b2f98af24bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:19.769768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ab5b-508c-49dd-9458-b2f98af24bdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:19.779329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32ab5b-508c-49dd-9458-b2f98af24bdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:19.781286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32ab5b-508c-49dd-9458-b2f98af24bdc None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-bb32ab5b-508c-49dd-9458-b2f98af24bdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:19.800734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:19.801714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:19.802650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:19.802650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:19.802650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:19.802650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:19.802650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:19.802650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:19.802650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:19.802650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:19.802650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 349/706] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:19.808367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ab5b-508c-49dd-9458-b2f98af24bdc None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:19.850485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916a549-4b64-4a78-be46-743a053e0cb9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:19.889972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab126b-e40a-417e-a87c-27d24432fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:19.891475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab126b-e40a-417e-a87c-27d24432fdf6 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:19.891969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab126b-e40a-417e-a87c-27d24432fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:19.892457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:19.893726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab126b-e40a-417e-a87c-27d24432fdf6 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:19.895321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab126b-e40a-417e-a87c-27d24432fdf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:19.896368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab126b-e40a-417e-a87c-27d24432fdf6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:19.904784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:19.918264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ab5b-508c-49dd-9458-b2f98af24bdc None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:19.929222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 358/707] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:50:19 2022] POST /identity/v3/users => generated 254 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:50:19.958893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8aa168-ba32-4f2c-9581-70d029e5e54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:19.959574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8aa168-ba32-4f2c-9581-70d029e5e54b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:19.959863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8aa168-ba32-4f2c-9581-70d029e5e54b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:19.960141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8aa168-ba32-4f2c-9581-70d029e5e54b None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:19.960941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8aa168-ba32-4f2c-9581-70d029e5e54b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:19.961287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8aa168-ba32-4f2c-9581-70d029e5e54b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:19.962905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8aa168-ba32-4f2c-9581-70d029e5e54b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:19.977578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 350/708] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:19.983491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab126b-e40a-417e-a87c-27d24432fdf6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:20.093483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11392ecd-62e1-45ee-b3cb-6c5395549d1b None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.093820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11392ecd-62e1-45ee-b3cb-6c5395549d1b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.094070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11392ecd-62e1-45ee-b3cb-6c5395549d1b None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.096159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 351/709] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:20.101919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04a469-d3ba-407b-b30d-7c7c7f62bf6f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.102218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04a469-d3ba-407b-b30d-7c7c7f62bf6f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.102505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04a469-d3ba-407b-b30d-7c7c7f62bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.114750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab126b-e40a-417e-a87c-27d24432fdf6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:20.137249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7f04a469-d3ba-407b-b30d-7c7c7f62bf6f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:20.138558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7f04a469-d3ba-407b-b30d-7c7c7f62bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:20.188112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106c8a9-fff1-4243-a259-0b2f11e14704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:20.189519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106c8a9-fff1-4243-a259-0b2f11e14704 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.189817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106c8a9-fff1-4243-a259-0b2f11e14704 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.190102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106c8a9-fff1-4243-a259-0b2f11e14704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.190954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106c8a9-fff1-4243-a259-0b2f11e14704 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=dcbcec65220c4ebb95f974925fa73f34) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:20.191275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106c8a9-fff1-4243-a259-0b2f11e14704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:20.192750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106c8a9-fff1-4243-a259-0b2f11e14704 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:20.201309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 359/710] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:19 2022] GET /identity/v3/role_assignments?user.id=dcbcec65220c4ebb95f974925fa73f34&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=e089a0d6040040c4bcf143a680488e3d => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:20.241926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 352/711] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:20.266152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106c8a9-fff1-4243-a259-0b2f11e14704 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:20.349840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:20.402174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106c8a9-fff1-4243-a259-0b2f11e14704 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:20.439962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a965f9d-6c98-4960-99f9-501ffc62e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:20.441130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965f9d-6c98-4960-99f9-501ffc62e976 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.441628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965f9d-6c98-4960-99f9-501ffc62e976 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.442107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965f9d-6c98-4960-99f9-501ffc62e976 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.442992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a965f9d-6c98-4960-99f9-501ffc62e976 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:20.444639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a965f9d-6c98-4960-99f9-501ffc62e976 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5a965f9d-6c98-4960-99f9-501ffc62e976 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:20.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:20.461026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:50:20.461987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:20.465140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 360/712] 10.222.0.70 () {56 vars in 1129 bytes} [Tue Jun 21 18:50:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:20.473114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04a469-d3ba-407b-b30d-7c7c7f62bf6f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:20.500439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b4142-3f15-43c1-bfe7-c3cb6c7da77d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.500751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b4142-3f15-43c1-bfe7-c3cb6c7da77d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.501005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b4142-3f15-43c1-bfe7-c3cb6c7da77d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.502737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 361/713] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:20.512442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afda7fe-35a7-448b-840c-d21dc02544c7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.512745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afda7fe-35a7-448b-840c-d21dc02544c7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.513008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afda7fe-35a7-448b-840c-d21dc02544c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.538832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2afda7fe-35a7-448b-840c-d21dc02544c7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:20.540123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2afda7fe-35a7-448b-840c-d21dc02544c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:20.576808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:20.633755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04a469-d3ba-407b-b30d-7c7c7f62bf6f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:20.678867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 362/714] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:50:20.703943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afda7fe-35a7-448b-840c-d21dc02544c7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:20.706471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea26c8-adf5-4729-8b35-56bb58574634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:20.707138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea26c8-adf5-4729-8b35-56bb58574634 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.707415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea26c8-adf5-4729-8b35-56bb58574634 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.707688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea26c8-adf5-4729-8b35-56bb58574634 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.708399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea26c8-adf5-4729-8b35-56bb58574634 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:20.708702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea26c8-adf5-4729-8b35-56bb58574634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:20.715890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea26c8-adf5-4729-8b35-56bb58574634 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:20.724111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 353/715] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jun 21 18:50:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:20.734143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea26c8-adf5-4729-8b35-56bb58574634 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:20.812483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:20.843716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea26c8-adf5-4729-8b35-56bb58574634 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:20.877985 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:20.884060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31128d9-ac08-4dbb-9d89-038ad004f034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:20.884736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31128d9-ac08-4dbb-9d89-038ad004f034 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.885051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31128d9-ac08-4dbb-9d89-038ad004f034 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.885372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31128d9-ac08-4dbb-9d89-038ad004f034 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.886093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31128d9-ac08-4dbb-9d89-038ad004f034 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:20.886400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31128d9-ac08-4dbb-9d89-038ad004f034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:20.887789 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31128d9-ac08-4dbb-9d89-038ad004f034 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:20.894137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 354/716] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jun 21 18:50:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:20.919945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afda7fe-35a7-448b-840c-d21dc02544c7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:20.961489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4739c-3be7-4839-9b29-3a1420ef0cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:20.962219 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4739c-3be7-4839-9b29-3a1420ef0cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:20.962505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4739c-3be7-4839-9b29-3a1420ef0cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:20.962783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4739c-3be7-4839-9b29-3a1420ef0cd4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:20.964011 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4739c-3be7-4839-9b29-3a1420ef0cd4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:20.965425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4739c-3be7-4839-9b29-3a1420ef0cd4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d1b4739c-3be7-4839-9b29-3a1420ef0cd4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:20.978408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:20.979562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:20.980416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 363/717] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:20 2022] GET /identity/v3/services/network => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:20.985958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31128d9-ac08-4dbb-9d89-038ad004f034 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:21.104764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31128d9-ac08-4dbb-9d89-038ad004f034 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:21.142327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf3c93-6f3d-4525-a0df-d8d9973bd53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:21.143119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3c93-6f3d-4525-a0df-d8d9973bd53a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:21.143289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3c93-6f3d-4525-a0df-d8d9973bd53a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:21.143574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3c93-6f3d-4525-a0df-d8d9973bd53a None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:21.144318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf3c93-6f3d-4525-a0df-d8d9973bd53a None admin] RBAC: Adding query filter params (name=network) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:21.144633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf3c93-6f3d-4525-a0df-d8d9973bd53a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:21.146105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf3c93-6f3d-4525-a0df-d8d9973bd53a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:21.152351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 355/718] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:21.158461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4739c-3be7-4839-9b29-3a1420ef0cd4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:21.272356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4739c-3be7-4839-9b29-3a1420ef0cd4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:21.286896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:21.324529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027adcb9-ff07-4739-8066-210519942cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:21.329110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027adcb9-ff07-4739-8066-210519942cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:21.329110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027adcb9-ff07-4739-8066-210519942cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:21.329110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027adcb9-ff07-4739-8066-210519942cd3 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:21.329110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027adcb9-ff07-4739-8066-210519942cd3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:21.329110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027adcb9-ff07-4739-8066-210519942cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:21.329110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027adcb9-ff07-4739-8066-210519942cd3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:21.334922 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 364/719] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:22.232702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6b23d-4637-4503-9d36-45e369aed0ec None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:22.233524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6b23d-4637-4503-9d36-45e369aed0ec None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:22.233524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6b23d-4637-4503-9d36-45e369aed0ec None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:22.235021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 356/720] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:22.240697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7ef59-7fee-4c61-b517-7e66ef3bf1e8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:22.241486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7ef59-7fee-4c61-b517-7e66ef3bf1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:22.241486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7ef59-7fee-4c61-b517-7e66ef3bf1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:22.258715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-93b7ef59-7fee-4c61-b517-7e66ef3bf1e8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:22.259956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-93b7ef59-7fee-4c61-b517-7e66ef3bf1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:22.371478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 365/721] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 18:50:22.420435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6b23d-4637-4503-9d36-45e369aed0ec None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:22.503598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:22.563129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6b23d-4637-4503-9d36-45e369aed0ec None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:22.609598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95aba2-b019-4ba3-a759-aed66a612d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:22.610421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95aba2-b019-4ba3-a759-aed66a612d8d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:22.610712 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95aba2-b019-4ba3-a759-aed66a612d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:22.610993 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95aba2-b019-4ba3-a759-aed66a612d8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:22.630530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95aba2-b019-4ba3-a759-aed66a612d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6f95aba2-b019-4ba3-a759-aed66a612d8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:22.639023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:22.640002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:22.640840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 357/722] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:22.647901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7ef59-7fee-4c61-b517-7e66ef3bf1e8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:22.805417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7ef59-7fee-4c61-b517-7e66ef3bf1e8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:22.806633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d141ce-8b2f-4228-8c71-bd02d707b2ca None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:22.807483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d141ce-8b2f-4228-8c71-bd02d707b2ca None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:22.807769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d141ce-8b2f-4228-8c71-bd02d707b2ca None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:22.809281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 358/723] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:22.818600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838c2f2-bb53-4dde-82ba-acbadc3b9836 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:22.818895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838c2f2-bb53-4dde-82ba-acbadc3b9836 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:22.819186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838c2f2-bb53-4dde-82ba-acbadc3b9836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:22.823623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:22.851474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2838c2f2-bb53-4dde-82ba-acbadc3b9836 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:22.854152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2838c2f2-bb53-4dde-82ba-acbadc3b9836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:22.880200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594726c-5bcf-464a-962c-24a94be51090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:22.881263 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594726c-5bcf-464a-962c-24a94be51090 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:22.881814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594726c-5bcf-464a-962c-24a94be51090 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:22.882296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594726c-5bcf-464a-962c-24a94be51090 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:22.883257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594726c-5bcf-464a-962c-24a94be51090 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:22.883798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594726c-5bcf-464a-962c-24a94be51090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:22.886005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594726c-5bcf-464a-962c-24a94be51090 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:22.895690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 366/724] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:22.905140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594726c-5bcf-464a-962c-24a94be51090 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:22.958177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 359/725] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:22.959577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9004780-0b37-4b65-8747-febb83ee0deb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:22.959867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9004780-0b37-4b65-8747-febb83ee0deb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:22.960254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9004780-0b37-4b65-8747-febb83ee0deb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:22.961950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 360/726] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:22.967304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6565-02f6-4f48-bd47-763785ef6c4c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:22.967591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6565-02f6-4f48-bd47-763785ef6c4c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:22.967868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6565-02f6-4f48-bd47-763785ef6c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.000166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-221b6565-02f6-4f48-bd47-763785ef6c4c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:23.001402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-221b6565-02f6-4f48-bd47-763785ef6c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:23.016270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594726c-5bcf-464a-962c-24a94be51090 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:23.059903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:23.071146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb064bf5-fbf9-42b9-a745-501e9149c4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:23.073118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb064bf5-fbf9-42b9-a745-501e9149c4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.073557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb064bf5-fbf9-42b9-a745-501e9149c4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.073982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb064bf5-fbf9-42b9-a745-501e9149c4b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.087566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 361/727] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:23.088906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6565-02f6-4f48-bd47-763785ef6c4c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.089866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb064bf5-fbf9-42b9-a745-501e9149c4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:23.091819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb064bf5-fbf9-42b9-a745-501e9149c4b3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-eb064bf5-fbf9-42b9-a745-501e9149c4b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:23.118746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:23.120435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:23.121506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:23.121506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:23.121506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:23.121506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:23.122830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 367/728] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:23.128980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb064bf5-fbf9-42b9-a745-501e9149c4b3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.158745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:23.225319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6565-02f6-4f48-bd47-763785ef6c4c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:23.263974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb064bf5-fbf9-42b9-a745-501e9149c4b3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:23.270336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edad8e-7570-45b0-96e0-9cfc5ee4d49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:23.271638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edad8e-7570-45b0-96e0-9cfc5ee4d49c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.272071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edad8e-7570-45b0-96e0-9cfc5ee4d49c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.272496 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edad8e-7570-45b0-96e0-9cfc5ee4d49c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.283755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04edad8e-7570-45b0-96e0-9cfc5ee4d49c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-04edad8e-7570-45b0-96e0-9cfc5ee4d49c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:23.292166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:23.293339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:23.294195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:23.296739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 362/729] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:23.297763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edad8e-7570-45b0-96e0-9cfc5ee4d49c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.306445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1309e35-9c21-4202-a769-7e7a8cd52ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:23.307809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1309e35-9c21-4202-a769-7e7a8cd52ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.308167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1309e35-9c21-4202-a769-7e7a8cd52ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.308568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1309e35-9c21-4202-a769-7e7a8cd52ef5 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.309424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1309e35-9c21-4202-a769-7e7a8cd52ef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:23.309804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1309e35-9c21-4202-a769-7e7a8cd52ef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:23.311856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1309e35-9c21-4202-a769-7e7a8cd52ef5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:23.331177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 368/730] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:23.331177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1309e35-9c21-4202-a769-7e7a8cd52ef5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.360497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:23.424920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edad8e-7570-45b0-96e0-9cfc5ee4d49c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:23.448783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:23.464730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29ab9b-d383-4928-92e3-f3daca64f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:23.465924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29ab9b-d383-4928-92e3-f3daca64f60b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.466415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29ab9b-d383-4928-92e3-f3daca64f60b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.466891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29ab9b-d383-4928-92e3-f3daca64f60b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.471098 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1309e35-9c21-4202-a769-7e7a8cd52ef5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:23.479781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29ab9b-d383-4928-92e3-f3daca64f60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3a29ab9b-d383-4928-92e3-f3daca64f60b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:23.488209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:23.489222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:23.490077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:23.492845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 363/731] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:23.493899 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29ab9b-d383-4928-92e3-f3daca64f60b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.510304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed29b6d-4dcc-4691-8b65-518e5fccaaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:23.510993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed29b6d-4dcc-4691-8b65-518e5fccaaeb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.511292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed29b6d-4dcc-4691-8b65-518e5fccaaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.511566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed29b6d-4dcc-4691-8b65-518e5fccaaeb None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.512328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed29b6d-4dcc-4691-8b65-518e5fccaaeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:23.512643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed29b6d-4dcc-4691-8b65-518e5fccaaeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:23.514202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed29b6d-4dcc-4691-8b65-518e5fccaaeb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:23.523294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 369/732] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:23.524142 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed29b6d-4dcc-4691-8b65-518e5fccaaeb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.607399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:23.611666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29ab9b-d383-4928-92e3-f3daca64f60b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:23.635021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed29b6d-4dcc-4691-8b65-518e5fccaaeb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:23.652253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ec903-a27a-4ee4-9018-24461c721dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:23.653494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ec903-a27a-4ee4-9018-24461c721dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.654020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ec903-a27a-4ee4-9018-24461c721dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.654525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ec903-a27a-4ee4-9018-24461c721dd9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.663282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ec903-a27a-4ee4-9018-24461c721dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:23.665057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ec903-a27a-4ee4-9018-24461c721dd9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:23.673354 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c7ac0-bfd9-4ac0-aea7-5191aa49fd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:23.674035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c7ac0-bfd9-4ac0-aea7-5191aa49fd4b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.674324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c7ac0-bfd9-4ac0-aea7-5191aa49fd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.674606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c7ac0-bfd9-4ac0-aea7-5191aa49fd4b None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.675367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c7ac0-bfd9-4ac0-aea7-5191aa49fd4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:23.675681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c7ac0-bfd9-4ac0-aea7-5191aa49fd4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:23.677070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c7ac0-bfd9-4ac0-aea7-5191aa49fd4b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-562ec903-a27a-4ee4-9018-24461c721dd9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:23.683488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:23.684451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:23.685451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:23.686666 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 370/733] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:23.687869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c7ac0-bfd9-4ac0-aea7-5191aa49fd4b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.693261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:23.693629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 364/734] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:23.695028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ec903-a27a-4ee4-9018-24461c721dd9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.801757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c7ac0-bfd9-4ac0-aea7-5191aa49fd4b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:23.810534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ec903-a27a-4ee4-9018-24461c721dd9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:23.838650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a337131-8f64-4700-9578-27dc87b08c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:23.839587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a337131-8f64-4700-9578-27dc87b08c1a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.839869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a337131-8f64-4700-9578-27dc87b08c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.840219 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a337131-8f64-4700-9578-27dc87b08c1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.846018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb5e13-fe97-4167-b742-494e71aef319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:23.846814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb5e13-fe97-4167-b742-494e71aef319 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.847099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb5e13-fe97-4167-b742-494e71aef319 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.847376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb5e13-fe97-4167-b742-494e71aef319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.852516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a337131-8f64-4700-9578-27dc87b08c1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:23.854587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a337131-8f64-4700-9578-27dc87b08c1a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:23.859766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb5e13-fe97-4167-b742-494e71aef319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:23.861175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb5e13-fe97-4167-b742-494e71aef319 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7a337131-8f64-4700-9578-27dc87b08c1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:23.872613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:23.873795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:23.874736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:23.874736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:23.874736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:23.874736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:23.874736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 371/735] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:23.874736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a337131-8f64-4700-9578-27dc87b08c1a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6adb5e13-fe97-4167-b742-494e71aef319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:23.883690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:23.884645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:23.886147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:23.886147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:23.886147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:23.886147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:23.886147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 365/736] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:23.887127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57415e-9c38-4c37-8e68-0f654c4d135b None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:23.887127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57415e-9c38-4c37-8e68-0f654c4d135b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:23.887127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57415e-9c38-4c37-8e68-0f654c4d135b None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:23.888532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 366/737] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:23.889117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57415e-9c38-4c37-8e68-0f654c4d135b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:23.905858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:23.945876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:24.001803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57415e-9c38-4c37-8e68-0f654c4d135b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.004771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a337131-8f64-4700-9578-27dc87b08c1a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.038303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071fb2b-47a1-4b9f-a157-25a5a3b8c093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.038964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071fb2b-47a1-4b9f-a157-25a5a3b8c093 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.039245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071fb2b-47a1-4b9f-a157-25a5a3b8c093 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.039519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071fb2b-47a1-4b9f-a157-25a5a3b8c093 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.040239 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071fb2b-47a1-4b9f-a157-25a5a3b8c093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:24.040545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071fb2b-47a1-4b9f-a157-25a5a3b8c093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.040958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67e055-5a16-4bf9-af6a-e3e35df970b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.042082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67e055-5a16-4bf9-af6a-e3e35df970b1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.042082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67e055-5a16-4bf9-af6a-e3e35df970b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.042287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071fb2b-47a1-4b9f-a157-25a5a3b8c093 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.042373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67e055-5a16-4bf9-af6a-e3e35df970b1 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.043079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67e055-5a16-4bf9-af6a-e3e35df970b1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:24.043397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67e055-5a16-4bf9-af6a-e3e35df970b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.044801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67e055-5a16-4bf9-af6a-e3e35df970b1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.053781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 367/738] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:24.054695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071fb2b-47a1-4b9f-a157-25a5a3b8c093 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:24.062868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 372/739] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jun 21 18:50:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:24.064367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31427aa-a50a-4ebd-b83b-b8814b0b5187 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.064663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31427aa-a50a-4ebd-b83b-b8814b0b5187 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.064945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31427aa-a50a-4ebd-b83b-b8814b0b5187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.082699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d31427aa-a50a-4ebd-b83b-b8814b0b5187 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:24.083920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d31427aa-a50a-4ebd-b83b-b8814b0b5187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:24.127502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:24.152058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 373/740] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:24.153706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47faa0c-7ae3-48ac-b3da-2dabe8d4f824 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.153998 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47faa0c-7ae3-48ac-b3da-2dabe8d4f824 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.154282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47faa0c-7ae3-48ac-b3da-2dabe8d4f824 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.155552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 374/741] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:24.156245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47faa0c-7ae3-48ac-b3da-2dabe8d4f824 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:24.162226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071fb2b-47a1-4b9f-a157-25a5a3b8c093 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.195906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c985f-f476-418a-bb37-8e851cc6a7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.196568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c985f-f476-418a-bb37-8e851cc6a7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.196860 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c985f-f476-418a-bb37-8e851cc6a7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.197177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c985f-f476-418a-bb37-8e851cc6a7d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.198445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c985f-f476-418a-bb37-8e851cc6a7d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:24.198756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c985f-f476-418a-bb37-8e851cc6a7d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.200138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c985f-f476-418a-bb37-8e851cc6a7d2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.212012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 368/742] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:24.212817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c985f-f476-418a-bb37-8e851cc6a7d2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:24.217469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:24.274687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47faa0c-7ae3-48ac-b3da-2dabe8d4f824 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.310199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e2e40-f5fc-4284-8425-980fd99d8253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.311022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e2e40-f5fc-4284-8425-980fd99d8253 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.311311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e2e40-f5fc-4284-8425-980fd99d8253 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.311591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e2e40-f5fc-4284-8425-980fd99d8253 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.314877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c985f-f476-418a-bb37-8e851cc6a7d2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.320192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e2e40-f5fc-4284-8425-980fd99d8253 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.324035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e2e40-f5fc-4284-8425-980fd99d8253 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b96e2e40-f5fc-4284-8425-980fd99d8253 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:24.338818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:24.339887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:24.340829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 375/743] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jun 21 18:50:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:24.341471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87e5d1-05a3-404a-a576-06bea15ef8f4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.341542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87e5d1-05a3-404a-a576-06bea15ef8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.341808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87e5d1-05a3-404a-a576-06bea15ef8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.348759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:24.354919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547906a9-73cc-4d9f-94a6-ce788ade7586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.355578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547906a9-73cc-4d9f-94a6-ce788ade7586 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.355870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547906a9-73cc-4d9f-94a6-ce788ade7586 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.356159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547906a9-73cc-4d9f-94a6-ce788ade7586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.358486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2d87e5d1-05a3-404a-a576-06bea15ef8f4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:24.359682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2d87e5d1-05a3-404a-a576-06bea15ef8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:24.423316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547906a9-73cc-4d9f-94a6-ce788ade7586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.424692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547906a9-73cc-4d9f-94a6-ce788ade7586 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.426234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 376/744] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 18:50:24.427056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87e5d1-05a3-404a-a576-06bea15ef8f4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:24.459644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:24.526378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87e5d1-05a3-404a-a576-06bea15ef8f4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.538022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 369/745] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 18:50:24.538854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547906a9-73cc-4d9f-94a6-ce788ade7586 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:24.566444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529592c-533f-4881-9cfc-2d3d5fc6703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.567113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529592c-533f-4881-9cfc-2d3d5fc6703d None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.567394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529592c-533f-4881-9cfc-2d3d5fc6703d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.567674 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529592c-533f-4881-9cfc-2d3d5fc6703d None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.568396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529592c-533f-4881-9cfc-2d3d5fc6703d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.569813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529592c-533f-4881-9cfc-2d3d5fc6703d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.570700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:24.583766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 377/746] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:50:24 2022] POST /identity/v3/services => generated 251 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:24.584825 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529592c-533f-4881-9cfc-2d3d5fc6703d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:24.666724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547906a9-73cc-4d9f-94a6-ce788ade7586 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.692134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:24.715011 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529592c-533f-4881-9cfc-2d3d5fc6703d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.720516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a7038-e818-4437-b074-f181410d527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.721382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7038-e818-4437-b074-f181410d527f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.721666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7038-e818-4437-b074-f181410d527f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.721938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7038-e818-4437-b074-f181410d527f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.730109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a7038-e818-4437-b074-f181410d527f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.731589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a7038-e818-4437-b074-f181410d527f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-709a7038-e818-4437-b074-f181410d527f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:24.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:24.748348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:24.749297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 370/747] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:50:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:24.749891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a7038-e818-4437-b074-f181410d527f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:24.761760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d569ca-8968-481d-b763-f564f4514913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.762448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d569ca-8968-481d-b763-f564f4514913 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.762740 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d569ca-8968-481d-b763-f564f4514913 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.763022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d569ca-8968-481d-b763-f564f4514913 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.764279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d569ca-8968-481d-b763-f564f4514913 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:24.764602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d569ca-8968-481d-b763-f564f4514913 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.766448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d569ca-8968-481d-b763-f564f4514913 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.785035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 378/748] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jun 21 18:50:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:24.786371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d569ca-8968-481d-b763-f564f4514913 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:24.860058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:24.875947 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a7038-e818-4437-b074-f181410d527f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.896053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d569ca-8968-481d-b763-f564f4514913 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:24.913962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eb8e6-2301-401b-8b20-ecf8dda3d81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.914632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eb8e6-2301-401b-8b20-ecf8dda3d81c None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.914921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eb8e6-2301-401b-8b20-ecf8dda3d81c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.915203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eb8e6-2301-401b-8b20-ecf8dda3d81c None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.915942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66eb8e6-2301-401b-8b20-ecf8dda3d81c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.917922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66eb8e6-2301-401b-8b20-ecf8dda3d81c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.934234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 371/749] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:50:24 2022] POST /identity/v3/services => generated 240 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:24.935066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eb8e6-2301-401b-8b20-ecf8dda3d81c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:24.947318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8094ba1-54b8-4c25-9bbb-983d82678e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:24.948442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8094ba1-54b8-4c25-9bbb-983d82678e04 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:24.948877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8094ba1-54b8-4c25-9bbb-983d82678e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:24.949339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8094ba1-54b8-4c25-9bbb-983d82678e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:24.962395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8094ba1-54b8-4c25-9bbb-983d82678e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:24.964500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8094ba1-54b8-4c25-9bbb-983d82678e04 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:24.968423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e8094ba1-54b8-4c25-9bbb-983d82678e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:24.988462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:24.989461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:24.990353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:24.990353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:24.990353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:24.990353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:24.992782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 379/750] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:24.993880 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8094ba1-54b8-4c25-9bbb-983d82678e04 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:25.113913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eb8e6-2301-401b-8b20-ecf8dda3d81c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:25.120069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8094ba1-54b8-4c25-9bbb-983d82678e04 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:25.156880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be908b-a628-4ce5-8347-f03b03028a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:25.157857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be908b-a628-4ce5-8347-f03b03028a81 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:25.158291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be908b-a628-4ce5-8347-f03b03028a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:25.158716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be908b-a628-4ce5-8347-f03b03028a81 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:25.159190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd802b-8eff-4a7f-9acb-0f4db7881704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:25.159881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be908b-a628-4ce5-8347-f03b03028a81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:25.160013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd802b-8eff-4a7f-9acb-0f4db7881704 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:25.160260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd802b-8eff-4a7f-9acb-0f4db7881704 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:25.160540 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd802b-8eff-4a7f-9acb-0f4db7881704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:25.160739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be908b-a628-4ce5-8347-f03b03028a81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:25.162353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be908b-a628-4ce5-8347-f03b03028a81 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:25.176861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd802b-8eff-4a7f-9acb-0f4db7881704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:25.178521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd802b-8eff-4a7f-9acb-0f4db7881704 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:25.185019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 372/751] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jun 21 18:50:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:25.185896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be908b-a628-4ce5-8347-f03b03028a81 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:25.196544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-eebd802b-8eff-4a7f-9acb-0f4db7881704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:25.199994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:25.201032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:25.202181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:25.202181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:25.202181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:25.202181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:25.204889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 380/752] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:25.205869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd802b-8eff-4a7f-9acb-0f4db7881704 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:25.273726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:25.305139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be908b-a628-4ce5-8347-f03b03028a81 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:25.350499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd802b-8eff-4a7f-9acb-0f4db7881704 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:25.357115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f41398-4333-4111-b29e-127abe72c693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:25.358308 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f41398-4333-4111-b29e-127abe72c693 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:25.359260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f41398-4333-4111-b29e-127abe72c693 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:25.359690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f41398-4333-4111-b29e-127abe72c693 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:25.361021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f41398-4333-4111-b29e-127abe72c693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:25.361503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f41398-4333-4111-b29e-127abe72c693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:25.363765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f41398-4333-4111-b29e-127abe72c693 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:25.381154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 373/753] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:25.382241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f41398-4333-4111-b29e-127abe72c693 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:25.392592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9e892-acb0-44e2-b315-e2dbaf46a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:25.393505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9e892-acb0-44e2-b315-e2dbaf46a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:25.393626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9e892-acb0-44e2-b315-e2dbaf46a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:25.393844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9e892-acb0-44e2-b315-e2dbaf46a2f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:25.473922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:25.475659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9e892-acb0-44e2-b315-e2dbaf46a2f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:25.477617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9e892-acb0-44e2-b315-e2dbaf46a2f6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:25.513524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f41398-4333-4111-b29e-127abe72c693 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:25.546421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:25.551139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca60c5-6478-4384-9278-768f17ba1148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:25.552618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca60c5-6478-4384-9278-768f17ba1148 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:25.553211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca60c5-6478-4384-9278-768f17ba1148 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:25.553451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca60c5-6478-4384-9278-768f17ba1148 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:25.554195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca60c5-6478-4384-9278-768f17ba1148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:25.554690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca60c5-6478-4384-9278-768f17ba1148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:25.556304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca60c5-6478-4384-9278-768f17ba1148 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:25.570449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 374/754] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:25.571291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca60c5-6478-4384-9278-768f17ba1148 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:25.615376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 381/755] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:50:25.616490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9e892-acb0-44e2-b315-e2dbaf46a2f6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:25.686576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca60c5-6478-4384-9278-768f17ba1148 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:25.716237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:25.731395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48632c6b-c3e1-4c30-8fef-5c86c7542dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:25.732116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48632c6b-c3e1-4c30-8fef-5c86c7542dce None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:25.732405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48632c6b-c3e1-4c30-8fef-5c86c7542dce None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:25.732685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48632c6b-c3e1-4c30-8fef-5c86c7542dce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:25.742358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48632c6b-c3e1-4c30-8fef-5c86c7542dce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:25.744861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48632c6b-c3e1-4c30-8fef-5c86c7542dce None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:25.747735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9e892-acb0-44e2-b315-e2dbaf46a2f6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-48632c6b-c3e1-4c30-8fef-5c86c7542dce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:25.761416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:25.762646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:25.763779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:25.763779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:25.763779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:25.763779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:25.763779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:25.763779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:25.763779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:25.763779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:25.766230 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 375/756] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:25.767454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48632c6b-c3e1-4c30-8fef-5c86c7542dce None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:25.793582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309909c-270d-44b7-908b-b8c11777fe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:25.794515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309909c-270d-44b7-908b-b8c11777fe12 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:25.794813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309909c-270d-44b7-908b-b8c11777fe12 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:25.795165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309909c-270d-44b7-908b-b8c11777fe12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:25.803130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309909c-270d-44b7-908b-b8c11777fe12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:25.804791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309909c-270d-44b7-908b-b8c11777fe12 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d309909c-270d-44b7-908b-b8c11777fe12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:25.820610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:25.821783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:25.822931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:25.822931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:25.822931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:25.822931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:25.822931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:25.822931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:25.822931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:25.822931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:25.828438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 382/757] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:25.829428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309909c-270d-44b7-908b-b8c11777fe12 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:25.842116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:25.886813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48632c6b-c3e1-4c30-8fef-5c86c7542dce None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:25.925022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d85212-c822-448a-8f61-6761cc4020dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:25.926095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d85212-c822-448a-8f61-6761cc4020dc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:25.926527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d85212-c822-448a-8f61-6761cc4020dc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:25.927094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d85212-c822-448a-8f61-6761cc4020dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:25.940101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309909c-270d-44b7-908b-b8c11777fe12 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:25.940411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d85212-c822-448a-8f61-6761cc4020dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:25.942198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d85212-c822-448a-8f61-6761cc4020dc None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-62d85212-c822-448a-8f61-6761cc4020dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:25.962525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:25.964180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:25.965116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:25.965116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:25.965116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:25.965116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:25.967076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 376/758] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:25.968400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d85212-c822-448a-8f61-6761cc4020dc None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:25.976343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a87d9-cf21-476d-bbb1-913cf42cf102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:25.977056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a87d9-cf21-476d-bbb1-913cf42cf102 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:25.977373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a87d9-cf21-476d-bbb1-913cf42cf102 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:25.977655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a87d9-cf21-476d-bbb1-913cf42cf102 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:25.978443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a87d9-cf21-476d-bbb1-913cf42cf102 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:25.979324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a87d9-cf21-476d-bbb1-913cf42cf102 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:25.980904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a87d9-cf21-476d-bbb1-913cf42cf102 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:25.994833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 383/759] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:25.995421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a87d9-cf21-476d-bbb1-913cf42cf102 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:26.014060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:26.069232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:26.083695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d85212-c822-448a-8f61-6761cc4020dc None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:26.109532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a87d9-cf21-476d-bbb1-913cf42cf102 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:26.125226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c7ca5-05e6-4749-95c3-e1d7e2ac1a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:26.126126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c7ca5-05e6-4749-95c3-e1d7e2ac1a1e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:26.126600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c7ca5-05e6-4749-95c3-e1d7e2ac1a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:26.127028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c7ca5-05e6-4749-95c3-e1d7e2ac1a1e None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:26.128157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c7ca5-05e6-4749-95c3-e1d7e2ac1a1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:26.128616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c7ca5-05e6-4749-95c3-e1d7e2ac1a1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:26.130256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c7ca5-05e6-4749-95c3-e1d7e2ac1a1e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:26.144268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 377/760] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:26.145349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c7ca5-05e6-4749-95c3-e1d7e2ac1a1e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:26.151217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5920c-522c-49b7-a69b-1ef82056674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:26.151896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5920c-522c-49b7-a69b-1ef82056674f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:26.152179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5920c-522c-49b7-a69b-1ef82056674f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:26.152457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5920c-522c-49b7-a69b-1ef82056674f None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:26.153220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5920c-522c-49b7-a69b-1ef82056674f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:26.153530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5920c-522c-49b7-a69b-1ef82056674f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:26.154980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5920c-522c-49b7-a69b-1ef82056674f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:26.169437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 384/761] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:26.170352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5920c-522c-49b7-a69b-1ef82056674f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:26.258056 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:26.263081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c7ca5-05e6-4749-95c3-e1d7e2ac1a1e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:26.284799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5920c-522c-49b7-a69b-1ef82056674f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:26.301578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01168c82-17a4-4f93-ac36-6ebe75652481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:26.302659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01168c82-17a4-4f93-ac36-6ebe75652481 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:26.303210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01168c82-17a4-4f93-ac36-6ebe75652481 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:26.303651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01168c82-17a4-4f93-ac36-6ebe75652481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:26.304664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01168c82-17a4-4f93-ac36-6ebe75652481 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:26.305158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01168c82-17a4-4f93-ac36-6ebe75652481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:26.306905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01168c82-17a4-4f93-ac36-6ebe75652481 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:26.313785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:26.315287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 378/762] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:26 2022] GET /identity/v3/role_assignments?user.id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=e089a0d6040040c4bcf143a680488e3d => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:26.316320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01168c82-17a4-4f93-ac36-6ebe75652481 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:26.324144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611dafa-8ed8-44bc-bee3-47d86d55b3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:26.325127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611dafa-8ed8-44bc-bee3-47d86d55b3d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:26.325416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611dafa-8ed8-44bc-bee3-47d86d55b3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:26.325697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611dafa-8ed8-44bc-bee3-47d86d55b3d5 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/dcbcec65220c4ebb95f974925fa73f34/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:26.328026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611dafa-8ed8-44bc-bee3-47d86d55b3d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=dcbcec65220c4ebb95f974925fa73f34, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:26.329393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611dafa-8ed8-44bc-bee3-47d86d55b3d5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:26.350144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 385/763] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:50:26 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/dcbcec65220c4ebb95f974925fa73f34/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:50:26.492284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01168c82-17a4-4f93-ac36-6ebe75652481 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:26.548603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e758b3-145b-4016-b66d-99b1f59b37b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:26.549387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e758b3-145b-4016-b66d-99b1f59b37b2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:26.549676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e758b3-145b-4016-b66d-99b1f59b37b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:26.549956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e758b3-145b-4016-b66d-99b1f59b37b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:26.563503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e758b3-145b-4016-b66d-99b1f59b37b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:26.564985 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e758b3-145b-4016-b66d-99b1f59b37b2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-18e758b3-145b-4016-b66d-99b1f59b37b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:26.583640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:26.584610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:26.585603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:26.585603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:26.585603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:26.585603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:26.585603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:26.585603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:26.585603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:26.585603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:26.588606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 379/764] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:26.592191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611dafa-8ed8-44bc-bee3-47d86d55b3d5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:26.711558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611dafa-8ed8-44bc-bee3-47d86d55b3d5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:26.756208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da646d-5b6e-4cec-93e3-fd56b1af39fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:26.757291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da646d-5b6e-4cec-93e3-fd56b1af39fd None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:26.757801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da646d-5b6e-4cec-93e3-fd56b1af39fd None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:26.758282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da646d-5b6e-4cec-93e3-fd56b1af39fd None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:26.759279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da646d-5b6e-4cec-93e3-fd56b1af39fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:26.759839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da646d-5b6e-4cec-93e3-fd56b1af39fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:26.761614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da646d-5b6e-4cec-93e3-fd56b1af39fd None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:26.776022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 386/765] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:26.781662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e758b3-145b-4016-b66d-99b1f59b37b2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:26.800156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b71e0-ce14-4cf8-abe7-aa837c7ece72 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:26.800757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b71e0-ce14-4cf8-abe7-aa837c7ece72 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:26.801644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b71e0-ce14-4cf8-abe7-aa837c7ece72 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:26.803640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 387/766] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:26.809244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cb9cf-e30f-4870-845b-bb07f4b1c5e1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:26.810008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cb9cf-e30f-4870-845b-bb07f4b1c5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:26.810491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cb9cf-e30f-4870-845b-bb07f4b1c5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:26.827389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:26.833793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-571cb9cf-e30f-4870-845b-bb07f4b1c5e1 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:26.835428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-571cb9cf-e30f-4870-845b-bb07f4b1c5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:26.890382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:26.923064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 388/767] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:26.949754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cb9cf-e30f-4870-845b-bb07f4b1c5e1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:27.008488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e758b3-145b-4016-b66d-99b1f59b37b2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:27.079984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42946ff-b0bb-41dc-9862-c1d622efb72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:27.080983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42946ff-b0bb-41dc-9862-c1d622efb72f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:27.081510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42946ff-b0bb-41dc-9862-c1d622efb72f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:27.081989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42946ff-b0bb-41dc-9862-c1d622efb72f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:27.084177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42946ff-b0bb-41dc-9862-c1d622efb72f None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=9fee9b6e04f24f96adb1a70306463a91) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:27.084713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42946ff-b0bb-41dc-9862-c1d622efb72f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:27.090483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42946ff-b0bb-41dc-9862-c1d622efb72f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:27.093701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cb9cf-e30f-4870-845b-bb07f4b1c5e1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:27.102507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 380/768] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:26 2022] GET /identity/v3/role_assignments?user.id=9fee9b6e04f24f96adb1a70306463a91&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:27.254637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52927d5-c81e-4da0-a6b3-d542ee54bd81 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:27.254967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52927d5-c81e-4da0-a6b3-d542ee54bd81 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:27.255199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52927d5-c81e-4da0-a6b3-d542ee54bd81 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:27.256745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 381/769] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:27.257701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3d9a9-0c1b-44f7-91c9-66202ad15a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:27.258827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3d9a9-0c1b-44f7-91c9-66202ad15a12 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:27.259333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3d9a9-0c1b-44f7-91c9-66202ad15a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:27.259869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3d9a9-0c1b-44f7-91c9-66202ad15a12 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:27.260791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3d9a9-0c1b-44f7-91c9-66202ad15a12 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:27.262459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3d9a9-0c1b-44f7-91c9-66202ad15a12 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:27.274714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759565e9-ee32-42c1-a17d-7e28f825dd01 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:27.275030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759565e9-ee32-42c1-a17d-7e28f825dd01 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:27.275265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759565e9-ee32-42c1-a17d-7e28f825dd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5ce3d9a9-0c1b-44f7-91c9-66202ad15a12 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:27.290170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:27.291741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:50:27.292585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:27.295679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 389/770] 10.222.0.70 () {56 vars in 1129 bytes} [Tue Jun 21 18:50:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:27.301949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3d9a9-0c1b-44f7-91c9-66202ad15a12 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:27.309782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-759565e9-ee32-42c1-a17d-7e28f825dd01 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:27.315608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-759565e9-ee32-42c1-a17d-7e28f825dd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:27.394315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:27.465115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 382/771] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:27.519161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3d9a9-0c1b-44f7-91c9-66202ad15a12 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:27.527499 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759565e9-ee32-42c1-a17d-7e28f825dd01 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:27.568564 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265cdca-c948-4fc7-808e-8d4b935993bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:27.569591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:27.575736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265cdca-c948-4fc7-808e-8d4b935993bc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:27.577707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265cdca-c948-4fc7-808e-8d4b935993bc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:27.578964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265cdca-c948-4fc7-808e-8d4b935993bc None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:27.580724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265cdca-c948-4fc7-808e-8d4b935993bc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:27.583172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265cdca-c948-4fc7-808e-8d4b935993bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:27.585349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265cdca-c948-4fc7-808e-8d4b935993bc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:27.606185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 390/772] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jun 21 18:50:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:27.611970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265cdca-c948-4fc7-808e-8d4b935993bc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:27.728210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759565e9-ee32-42c1-a17d-7e28f825dd01 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:27.766270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d7496-32ae-4d73-8bec-17dddaabe08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:27.767106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d7496-32ae-4d73-8bec-17dddaabe08b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:27.767393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d7496-32ae-4d73-8bec-17dddaabe08b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:27.767674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d7496-32ae-4d73-8bec-17dddaabe08b None admin] PATH_INFO: `/v3/services/network` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:27.768342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d7496-32ae-4d73-8bec-17dddaabe08b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:27.769809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d7496-32ae-4d73-8bec-17dddaabe08b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3a5d7496-32ae-4d73-8bec-17dddaabe08b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:27.793251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:27.794316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:27.795175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 383/773] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:27 2022] GET /identity/v3/services/network => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:27.803667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d7496-32ae-4d73-8bec-17dddaabe08b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:27.917548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:27.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265cdca-c948-4fc7-808e-8d4b935993bc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:27.935193 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:27.938325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d7496-32ae-4d73-8bec-17dddaabe08b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:27.976769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de503158-4707-45a7-9606-4de2ecfc69d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:27.977507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de503158-4707-45a7-9606-4de2ecfc69d4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:27.977800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de503158-4707-45a7-9606-4de2ecfc69d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:27.978083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de503158-4707-45a7-9606-4de2ecfc69d4 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:27.979498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de503158-4707-45a7-9606-4de2ecfc69d4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:27.979762 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de503158-4707-45a7-9606-4de2ecfc69d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:27.981294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de503158-4707-45a7-9606-4de2ecfc69d4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:27.986791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 384/774] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:27.992140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de503158-4707-45a7-9606-4de2ecfc69d4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:28.003497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147dec3-8ac3-4927-94c2-0f10a4f6603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:28.004648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147dec3-8ac3-4927-94c2-0f10a4f6603c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:28.005172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147dec3-8ac3-4927-94c2-0f10a4f6603c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:28.005466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147dec3-8ac3-4927-94c2-0f10a4f6603c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:28.006186 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147dec3-8ac3-4927-94c2-0f10a4f6603c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:28.006499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147dec3-8ac3-4927-94c2-0f10a4f6603c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:28.011047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147dec3-8ac3-4927-94c2-0f10a4f6603c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:28.025154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 391/775] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jun 21 18:50:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:28.028930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147dec3-8ac3-4927-94c2-0f10a4f6603c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:28.130688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de503158-4707-45a7-9606-4de2ecfc69d4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:28.169024 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98773004-db8b-49f5-9d12-6c7183431ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:28.169901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98773004-db8b-49f5-9d12-6c7183431ecb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:28.170188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98773004-db8b-49f5-9d12-6c7183431ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:28.170467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98773004-db8b-49f5-9d12-6c7183431ecb None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:28.171216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98773004-db8b-49f5-9d12-6c7183431ecb None admin] RBAC: Adding query filter params (type=network) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:28.171527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98773004-db8b-49f5-9d12-6c7183431ecb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:28.172923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98773004-db8b-49f5-9d12-6c7183431ecb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:28.179028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 385/776] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:28.190242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98773004-db8b-49f5-9d12-6c7183431ecb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:28.225455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:28.232288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147dec3-8ac3-4927-94c2-0f10a4f6603c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:28.276701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd9e3a-cf7c-4584-827d-f6c8d00a8e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:28.277506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9e3a-cf7c-4584-827d-f6c8d00a8e25 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:28.277802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9e3a-cf7c-4584-827d-f6c8d00a8e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:28.278147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9e3a-cf7c-4584-827d-f6c8d00a8e25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:28.279017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd9e3a-cf7c-4584-827d-f6c8d00a8e25 None admin] RBAC: Adding query filter params (interface=public, service_id=ff8a3ec44d2f42cc9047dc6dc50095bb, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:28.279339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd9e3a-cf7c-4584-827d-f6c8d00a8e25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:28.280720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd9e3a-cf7c-4584-827d-f6c8d00a8e25 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:28.294484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 392/777] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jun 21 18:50:28 2022] GET /identity/v3/endpoints?service_id=ff8a3ec44d2f42cc9047dc6dc50095bb&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:28.295427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd9e3a-cf7c-4584-827d-f6c8d00a8e25 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:28.356846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:28.409585 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98773004-db8b-49f5-9d12-6c7183431ecb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:28.428064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd9e3a-cf7c-4584-827d-f6c8d00a8e25 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:28.465386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad079a-a26d-422b-82ac-ed065bb7ee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:28.466250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad079a-a26d-422b-82ac-ed065bb7ee0c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:28.466641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad079a-a26d-422b-82ac-ed065bb7ee0c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:28.467023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad079a-a26d-422b-82ac-ed065bb7ee0c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:28.467885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad079a-a26d-422b-82ac-ed065bb7ee0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:28.469464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad079a-a26d-422b-82ac-ed065bb7ee0c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:28.475555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 393/778] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jun 21 18:50:28 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:28.489017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9cb7b-7e05-4697-bdf5-002165e7d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:28.489824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9cb7b-7e05-4697-bdf5-002165e7d055 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:28.490224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9cb7b-7e05-4697-bdf5-002165e7d055 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:28.490609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9cb7b-7e05-4697-bdf5-002165e7d055 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:28.491528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9cb7b-7e05-4697-bdf5-002165e7d055 None admin] RBAC: Adding query filter params (interface=public, service_id=442e547a185543e8a1c337d9bedd815b, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:28.491996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9cb7b-7e05-4697-bdf5-002165e7d055 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:28.493583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9cb7b-7e05-4697-bdf5-002165e7d055 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:28.508449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 386/779] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jun 21 18:50:28 2022] GET /identity/v3/endpoints?service_id=442e547a185543e8a1c337d9bedd815b&interface=public®ion_id=RegionOne => generated 194 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:28.514478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad079a-a26d-422b-82ac-ed065bb7ee0c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:28.665022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad079a-a26d-422b-82ac-ed065bb7ee0c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:28.677620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:28.719946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e0c42-fada-4601-b551-932ec7cea185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:28.720618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e0c42-fada-4601-b551-932ec7cea185 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:28.720907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e0c42-fada-4601-b551-932ec7cea185 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:28.721226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e0c42-fada-4601-b551-932ec7cea185 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:28.721972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e0c42-fada-4601-b551-932ec7cea185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:28.723400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e0c42-fada-4601-b551-932ec7cea185 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:28.729614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 394/780] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jun 21 18:50:28 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:28.823506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c5ae-4161-4f92-b11c-aeef22a43ee2 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:28.824000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c5ae-4161-4f92-b11c-aeef22a43ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:28.824000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c5ae-4161-4f92-b11c-aeef22a43ee2 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:28.825694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 387/781] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:28.831411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad4bfb-69ad-4991-86c3-1b332274e4e2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:28.831710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad4bfb-69ad-4991-86c3-1b332274e4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:28.831986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad4bfb-69ad-4991-86c3-1b332274e4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:28.849885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-22ad4bfb-69ad-4991-86c3-1b332274e4e2 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:28.851159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-22ad4bfb-69ad-4991-86c3-1b332274e4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:28.928797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 395/782] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:28.954047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631c5ae-4161-4f92-b11c-aeef22a43ee2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:29.034493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:29.079724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7f299-19a3-4488-b7de-2aa100d65706 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.080026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7f299-19a3-4488-b7de-2aa100d65706 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.080214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7f299-19a3-4488-b7de-2aa100d65706 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.081776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 396/783] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:29.088944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631c5ae-4161-4f92-b11c-aeef22a43ee2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:29.089284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1278274-e07c-414d-8306-cb5901ed4478 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.089284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1278274-e07c-414d-8306-cb5901ed4478 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.089284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1278274-e07c-414d-8306-cb5901ed4478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.118086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c1278274-e07c-414d-8306-cb5901ed4478 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:29.121342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c1278274-e07c-414d-8306-cb5901ed4478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:29.160532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467e9eb-ec3c-42c5-85df-dd2b79daad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:29.161364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467e9eb-ec3c-42c5-85df-dd2b79daad46 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.161647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467e9eb-ec3c-42c5-85df-dd2b79daad46 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.161920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467e9eb-ec3c-42c5-85df-dd2b79daad46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.193678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467e9eb-ec3c-42c5-85df-dd2b79daad46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-e467e9eb-ec3c-42c5-85df-dd2b79daad46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:29.211460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:29.212480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:29.213424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:29.215386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 388/784] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:29.222077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467e9eb-ec3c-42c5-85df-dd2b79daad46 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:29.253357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 397/785] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:29.288490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1278274-e07c-414d-8306-cb5901ed4478 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:29.344296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:29.361390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467e9eb-ec3c-42c5-85df-dd2b79daad46 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:29.396700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:29.404635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507f4a1-897c-4dbc-ba49-4c477afef6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:29.405918 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507f4a1-897c-4dbc-ba49-4c477afef6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.406589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507f4a1-897c-4dbc-ba49-4c477afef6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.407138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507f4a1-897c-4dbc-ba49-4c477afef6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.408279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507f4a1-897c-4dbc-ba49-4c477afef6e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:29.408823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507f4a1-897c-4dbc-ba49-4c477afef6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:29.410518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507f4a1-897c-4dbc-ba49-4c477afef6e8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:29.423073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 389/786] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:29.428744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507f4a1-897c-4dbc-ba49-4c477afef6e8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:29.534763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507f4a1-897c-4dbc-ba49-4c477afef6e8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:29.572649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9bb19-1745-4884-a9bb-b46c7bcae720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:29.573862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9bb19-1745-4884-a9bb-b46c7bcae720 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.574384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9bb19-1745-4884-a9bb-b46c7bcae720 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.574934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9bb19-1745-4884-a9bb-b46c7bcae720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.598266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9bb19-1745-4884-a9bb-b46c7bcae720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:29.600880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9bb19-1745-4884-a9bb-b46c7bcae720 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:29.602965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1278274-e07c-414d-8306-cb5901ed4478 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-94e9bb19-1745-4884-a9bb-b46c7bcae720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:29.631438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:29.632435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:29.633382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:29.633382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:29.633382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:29.633382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:29.633382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 390/787] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:29.634409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4fcc9-34c3-4e4a-87ad-42d24cd431d3 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.634748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4fcc9-34c3-4e4a-87ad-42d24cd431d3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.635036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4fcc9-34c3-4e4a-87ad-42d24cd431d3 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.636597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 391/788] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:29.640750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4fcc9-34c3-4e4a-87ad-42d24cd431d3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:29.659035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6583a-e9d0-4c59-af6a-77f2e20d79ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:29.660912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6583a-e9d0-4c59-af6a-77f2e20d79ce None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.661411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6583a-e9d0-4c59-af6a-77f2e20d79ce None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.661836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6583a-e9d0-4c59-af6a-77f2e20d79ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.681572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6583a-e9d0-4c59-af6a-77f2e20d79ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-15d6583a-e9d0-4c59-af6a-77f2e20d79ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:29.692229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:29.693372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:29.694361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:29.702092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 398/789] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:29.702706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88f495-042b-4756-9793-748c83989577 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.703007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88f495-042b-4756-9793-748c83989577 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.703294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88f495-042b-4756-9793-748c83989577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.708420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:29.725207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2f88f495-042b-4756-9793-748c83989577 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:29.726428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2f88f495-042b-4756-9793-748c83989577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:29.773137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4fcc9-34c3-4e4a-87ad-42d24cd431d3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:29.814141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852beaaa-cb77-43cd-8a55-35ce3f11b85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:29.821109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 399/790] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:29.821263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852beaaa-cb77-43cd-8a55-35ce3f11b85c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.821359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88f495-042b-4756-9793-748c83989577 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:29.822033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852beaaa-cb77-43cd-8a55-35ce3f11b85c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.822559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852beaaa-cb77-43cd-8a55-35ce3f11b85c None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.823577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852beaaa-cb77-43cd-8a55-35ce3f11b85c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:29.824541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852beaaa-cb77-43cd-8a55-35ce3f11b85c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:29.826320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852beaaa-cb77-43cd-8a55-35ce3f11b85c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:29.833120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:29.854957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 392/791] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:29.858411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852beaaa-cb77-43cd-8a55-35ce3f11b85c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:29.936416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88f495-042b-4756-9793-748c83989577 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:29.975276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd65552-b254-45a6-ba14-11da2c1208a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:29.976326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd65552-b254-45a6-ba14-11da2c1208a6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:29.976884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd65552-b254-45a6-ba14-11da2c1208a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:29.977397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd65552-b254-45a6-ba14-11da2c1208a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:29.978410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd65552-b254-45a6-ba14-11da2c1208a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:29.978981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd65552-b254-45a6-ba14-11da2c1208a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:29.980581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd65552-b254-45a6-ba14-11da2c1208a6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:29.988400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852beaaa-cb77-43cd-8a55-35ce3f11b85c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:29.995067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 400/792] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:29.996311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd65552-b254-45a6-ba14-11da2c1208a6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:29.996818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:30.034016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e7d7-09e0-4908-8d5f-1c7abeda1015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:30.035239 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e7d7-09e0-4908-8d5f-1c7abeda1015 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:30.035804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e7d7-09e0-4908-8d5f-1c7abeda1015 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:30.036433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e7d7-09e0-4908-8d5f-1c7abeda1015 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:30.050052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1e7d7-09e0-4908-8d5f-1c7abeda1015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-7db1e7d7-09e0-4908-8d5f-1c7abeda1015 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:30.058449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:30.059477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:30.060329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:30.064286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 393/793] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:30.065528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e7d7-09e0-4908-8d5f-1c7abeda1015 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:30.103080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:30.132424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd65552-b254-45a6-ba14-11da2c1208a6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:30.175373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1740f-518d-429a-a13a-fbe318473ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:30.176579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1740f-518d-429a-a13a-fbe318473ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:30.177161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1740f-518d-429a-a13a-fbe318473ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:30.177692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1740f-518d-429a-a13a-fbe318473ae7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:30.188840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1740f-518d-429a-a13a-fbe318473ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:30.190858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1740f-518d-429a-a13a-fbe318473ae7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:30.207638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e7d7-09e0-4908-8d5f-1c7abeda1015 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-35f1740f-518d-429a-a13a-fbe318473ae7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:30.211758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:30.212969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:30.214706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:30.217016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 401/794] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:30.218065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1740f-518d-429a-a13a-fbe318473ae7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:30.248624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67862eb3-fc1b-45c8-8d15-bc1fc5325eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:30.249611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67862eb3-fc1b-45c8-8d15-bc1fc5325eae None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:30.250059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67862eb3-fc1b-45c8-8d15-bc1fc5325eae None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:30.250463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67862eb3-fc1b-45c8-8d15-bc1fc5325eae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:30.262801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67862eb3-fc1b-45c8-8d15-bc1fc5325eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:30.264474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67862eb3-fc1b-45c8-8d15-bc1fc5325eae None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-67862eb3-fc1b-45c8-8d15-bc1fc5325eae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:30.283481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:30.284593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:30.285839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:30.285839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:30.285839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:30.285839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:30.291674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 394/795] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:30.291674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67862eb3-fc1b-45c8-8d15-bc1fc5325eae None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:30.310762 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:30.358988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1740f-518d-429a-a13a-fbe318473ae7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:30.406926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6503c-636d-46ed-ab4a-83038e1ecd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:30.407518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6503c-636d-46ed-ab4a-83038e1ecd46 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:30.407801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6503c-636d-46ed-ab4a-83038e1ecd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:30.408078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6503c-636d-46ed-ab4a-83038e1ecd46 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:30.408828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6503c-636d-46ed-ab4a-83038e1ecd46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:30.409169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6503c-636d-46ed-ab4a-83038e1ecd46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:30.410554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6503c-636d-46ed-ab4a-83038e1ecd46 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:30.412558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:30.421882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67862eb3-fc1b-45c8-8d15-bc1fc5325eae None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:30.424011 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 402/796] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:30.425157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6503c-636d-46ed-ab4a-83038e1ecd46 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:30.464587 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b818f3-cd77-40f7-90ed-9476d3fd2628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:30.465352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b818f3-cd77-40f7-90ed-9476d3fd2628 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:30.465640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b818f3-cd77-40f7-90ed-9476d3fd2628 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:30.465919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b818f3-cd77-40f7-90ed-9476d3fd2628 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:30.466840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b818f3-cd77-40f7-90ed-9476d3fd2628 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:30.467133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b818f3-cd77-40f7-90ed-9476d3fd2628 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:30.468635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b818f3-cd77-40f7-90ed-9476d3fd2628 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:30.487279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 395/797] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jun 21 18:50:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:30.488268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b818f3-cd77-40f7-90ed-9476d3fd2628 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:30.564733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6503c-636d-46ed-ab4a-83038e1ecd46 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:30.570722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:30.597796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b818f3-cd77-40f7-90ed-9476d3fd2628 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:30.641355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f22cbc-e929-49d1-9216-fbc27d8f9838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:30.642562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22cbc-e929-49d1-9216-fbc27d8f9838 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:30.643088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22cbc-e929-49d1-9216-fbc27d8f9838 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:30.643589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22cbc-e929-49d1-9216-fbc27d8f9838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:30.656059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f22cbc-e929-49d1-9216-fbc27d8f9838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:30.657992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f22cbc-e929-49d1-9216-fbc27d8f9838 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-f8f22cbc-e929-49d1-9216-fbc27d8f9838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:30.678512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:30.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:30.680390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:30.680390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:30.680390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:30.680390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:30.684196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 396/798] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:30.685314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f22cbc-e929-49d1-9216-fbc27d8f9838 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:30.732646 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e61f11-4998-4214-9226-2fbe44f4a934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:30.733446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e61f11-4998-4214-9226-2fbe44f4a934 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:30.733739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e61f11-4998-4214-9226-2fbe44f4a934 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:30.734018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e61f11-4998-4214-9226-2fbe44f4a934 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:30.734798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e61f11-4998-4214-9226-2fbe44f4a934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:30.735112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e61f11-4998-4214-9226-2fbe44f4a934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:30.736477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e61f11-4998-4214-9226-2fbe44f4a934 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:30.753729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 403/799] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:30.753729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e61f11-4998-4214-9226-2fbe44f4a934 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:30.816201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f22cbc-e929-49d1-9216-fbc27d8f9838 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:30.821976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:30.854734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:30.865735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8db1da-b0f5-404a-9e44-319eb8b91e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:30.866923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8db1da-b0f5-404a-9e44-319eb8b91e66 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:30.867715 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8db1da-b0f5-404a-9e44-319eb8b91e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:30.868187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8db1da-b0f5-404a-9e44-319eb8b91e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:30.871275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e61f11-4998-4214-9226-2fbe44f4a934 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:30.882024 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8db1da-b0f5-404a-9e44-319eb8b91e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:30.884353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8db1da-b0f5-404a-9e44-319eb8b91e66 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:30.907022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef85ebd-070b-4e06-baa9-de02deb89cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-cd8db1da-b0f5-404a-9e44-319eb8b91e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:30.907697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:30.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:30.909617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:30.909617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:30.909617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:30.909617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:30.909841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef85ebd-070b-4e06-baa9-de02deb89cde None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:30.909841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef85ebd-070b-4e06-baa9-de02deb89cde None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:30.909841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef85ebd-070b-4e06-baa9-de02deb89cde None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:30.909841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef85ebd-070b-4e06-baa9-de02deb89cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:30.909841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef85ebd-070b-4e06-baa9-de02deb89cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:30.911726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef85ebd-070b-4e06-baa9-de02deb89cde None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:30.912351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 397/800] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:30.913601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8db1da-b0f5-404a-9e44-319eb8b91e66 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:30.929882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 404/801] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:30.930894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef85ebd-070b-4e06-baa9-de02deb89cde None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.038065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8db1da-b0f5-404a-9e44-319eb8b91e66 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.047527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef85ebd-070b-4e06-baa9-de02deb89cde None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.064943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:31.075678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f40f44-f4d8-4113-8d00-38c4cbe2436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.076473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f40f44-f4d8-4113-8d00-38c4cbe2436d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.076754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f40f44-f4d8-4113-8d00-38c4cbe2436d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.077034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f40f44-f4d8-4113-8d00-38c4cbe2436d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.083614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5c7c3-7d9f-46a8-9929-b1022565bf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.084299 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5c7c3-7d9f-46a8-9929-b1022565bf26 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.084424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f40f44-f4d8-4113-8d00-38c4cbe2436d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.084582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5c7c3-7d9f-46a8-9929-b1022565bf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.084857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5c7c3-7d9f-46a8-9929-b1022565bf26 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.085610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5c7c3-7d9f-46a8-9929-b1022565bf26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:31.085852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f40f44-f4d8-4113-8d00-38c4cbe2436d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.085960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5c7c3-7d9f-46a8-9929-b1022565bf26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.087251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5c7c3-7d9f-46a8-9929-b1022565bf26 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.099475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 405/802] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:31.100286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5c7c3-7d9f-46a8-9929-b1022565bf26 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-31f40f44-f4d8-4113-8d00-38c4cbe2436d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:31.101361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:31.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:50:31.103246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:31.105251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 398/803] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jun 21 18:50:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:31.106581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856866a1-db6f-4a92-877a-281941490c87 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.106914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856866a1-db6f-4a92-877a-281941490c87 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.107200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856866a1-db6f-4a92-877a-281941490c87 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.108506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 399/804] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:31.109608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a963e-a0a8-4dc7-b923-433253c99a29 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.109895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a963e-a0a8-4dc7-b923-433253c99a29 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.110171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a963e-a0a8-4dc7-b923-433253c99a29 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.111609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 400/805] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:31.112110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a963e-a0a8-4dc7-b923-433253c99a29 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.156075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:31.207137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5c7c3-7d9f-46a8-9929-b1022565bf26 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.216064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a963e-a0a8-4dc7-b923-433253c99a29 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.243030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37884df9-049f-4cc4-b8bb-e31db8fc0547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.243843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37884df9-049f-4cc4-b8bb-e31db8fc0547 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.244159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37884df9-049f-4cc4-b8bb-e31db8fc0547 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.244438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37884df9-049f-4cc4-b8bb-e31db8fc0547 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.251221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c07fb-1241-4a62-9535-ca4ccf1e832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.252017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c07fb-1241-4a62-9535-ca4ccf1e832b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.252205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37884df9-049f-4cc4-b8bb-e31db8fc0547 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.252334 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c07fb-1241-4a62-9535-ca4ccf1e832b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.252579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c07fb-1241-4a62-9535-ca4ccf1e832b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.253701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37884df9-049f-4cc4-b8bb-e31db8fc0547 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.259904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c07fb-1241-4a62-9535-ca4ccf1e832b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.261399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c07fb-1241-4a62-9535-ca4ccf1e832b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-37884df9-049f-4cc4-b8bb-e31db8fc0547 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:31.268823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:31.270016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 406/806] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:50:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:31.271069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37884df9-049f-4cc4-b8bb-e31db8fc0547 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-395c07fb-1241-4a62-9535-ca4ccf1e832b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:31.276158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:31.277248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 401/807] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:31.278672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69950d8d-fc44-4fa4-8f6d-3da079f06ba4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.279364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69950d8d-fc44-4fa4-8f6d-3da079f06ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.279364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69950d8d-fc44-4fa4-8f6d-3da079f06ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.296001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-69950d8d-fc44-4fa4-8f6d-3da079f06ba4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:31.297237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-69950d8d-fc44-4fa4-8f6d-3da079f06ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:31.314603 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:31.365124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 402/808] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:31.366422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe077e5-a173-4237-8db6-bfe448105014 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.366713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe077e5-a173-4237-8db6-bfe448105014 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.366994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe077e5-a173-4237-8db6-bfe448105014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.382730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9fe077e5-a173-4237-8db6-bfe448105014 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:31.384136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9fe077e5-a173-4237-8db6-bfe448105014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:31.390753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37884df9-049f-4cc4-b8bb-e31db8fc0547 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.407610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:31.426202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199715fd-4859-4621-9219-584f188d1ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.426873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199715fd-4859-4621-9219-584f188d1ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.427172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199715fd-4859-4621-9219-584f188d1ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.427458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199715fd-4859-4621-9219-584f188d1ea3 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.428288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199715fd-4859-4621-9219-584f188d1ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:31.428601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199715fd-4859-4621-9219-584f188d1ea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.430048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199715fd-4859-4621-9219-584f188d1ea3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.449241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 407/809] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jun 21 18:50:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:31.450103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199715fd-4859-4621-9219-584f188d1ea3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.453396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 403/810] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:31.454380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe077e5-a173-4237-8db6-bfe448105014 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.544987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:31.552611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199715fd-4859-4621-9219-584f188d1ea3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.557998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe077e5-a173-4237-8db6-bfe448105014 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.587240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651daaa-a5f1-4504-84a5-685969572597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.587910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651daaa-a5f1-4504-84a5-685969572597 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.588194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651daaa-a5f1-4504-84a5-685969572597 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.588475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651daaa-a5f1-4504-84a5-685969572597 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.589338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651daaa-a5f1-4504-84a5-685969572597 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:31.589671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651daaa-a5f1-4504-84a5-685969572597 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.591090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651daaa-a5f1-4504-84a5-685969572597 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.594511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcf4d0-218b-4d5c-9926-d092d3afa99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.595162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcf4d0-218b-4d5c-9926-d092d3afa99a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.595442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcf4d0-218b-4d5c-9926-d092d3afa99a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.595716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcf4d0-218b-4d5c-9926-d092d3afa99a None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.596497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcf4d0-218b-4d5c-9926-d092d3afa99a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:31.596813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcf4d0-218b-4d5c-9926-d092d3afa99a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.598267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcf4d0-218b-4d5c-9926-d092d3afa99a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.608376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 408/811] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jun 21 18:50:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:31.609235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651daaa-a5f1-4504-84a5-685969572597 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.615197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 404/812] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:31.615197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcf4d0-218b-4d5c-9926-d092d3afa99a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.636504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:31.717310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651daaa-a5f1-4504-84a5-685969572597 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.720518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcf4d0-218b-4d5c-9926-d092d3afa99a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.750941 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335c45e-3f0c-4bcf-9223-4096f8244870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.751727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c45e-3f0c-4bcf-9223-4096f8244870 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.752009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c45e-3f0c-4bcf-9223-4096f8244870 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.752292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c45e-3f0c-4bcf-9223-4096f8244870 None admin] PATH_INFO: `/v3/services/network` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.752936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335c45e-3f0c-4bcf-9223-4096f8244870 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.754327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335c45e-3f0c-4bcf-9223-4096f8244870 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.755222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f5c80-1b66-4306-bbb4-4658b2116bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.755872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f5c80-1b66-4306-bbb4-4658b2116bad None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.756169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f5c80-1b66-4306-bbb4-4658b2116bad None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.756452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f5c80-1b66-4306-bbb4-4658b2116bad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b335c45e-3f0c-4bcf-9223-4096f8244870 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:31.765977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:31.767007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:31.767849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 409/813] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:31 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:31.769436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335c45e-3f0c-4bcf-9223-4096f8244870 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.790325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:31.826556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f5c80-1b66-4306-bbb4-4658b2116bad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.827952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f5c80-1b66-4306-bbb4-4658b2116bad None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.852587 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:31.874476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335c45e-3f0c-4bcf-9223-4096f8244870 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:31.908771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442db95d-3a00-4099-8d22-79bf87d64047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:31.909609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db95d-3a00-4099-8d22-79bf87d64047 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:31.909892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db95d-3a00-4099-8d22-79bf87d64047 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:31.910171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db95d-3a00-4099-8d22-79bf87d64047 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:31.910851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442db95d-3a00-4099-8d22-79bf87d64047 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:31.912221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442db95d-3a00-4099-8d22-79bf87d64047 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-442db95d-3a00-4099-8d22-79bf87d64047 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:31.924761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:31.925952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:31.927189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 410/814] 10.222.0.70 () {56 vars in 1129 bytes} [Tue Jun 21 18:50:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:31.928079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442db95d-3a00-4099-8d22-79bf87d64047 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:31.943520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 405/815] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:50:31.944375 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f5c80-1b66-4306-bbb4-4658b2116bad None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.014552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:32.034384 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442db95d-3a00-4099-8d22-79bf87d64047 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.045730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f5c80-1b66-4306-bbb4-4658b2116bad None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.066488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:32.069256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c33d45-6de7-4374-977f-82bb8cb4a789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.070100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c33d45-6de7-4374-977f-82bb8cb4a789 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.070336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c33d45-6de7-4374-977f-82bb8cb4a789 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.070612 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c33d45-6de7-4374-977f-82bb8cb4a789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.081601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c33d45-6de7-4374-977f-82bb8cb4a789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.081738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992a7a4-1c85-4041-8708-0d94c6b29555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.082440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992a7a4-1c85-4041-8708-0d94c6b29555 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.082733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992a7a4-1c85-4041-8708-0d94c6b29555 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.082972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c33d45-6de7-4374-977f-82bb8cb4a789 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.083082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992a7a4-1c85-4041-8708-0d94c6b29555 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/ad2a56d3235b46e3b1fbb8bf0c2b4e0b/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.084938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992a7a4-1c85-4041-8708-0d94c6b29555 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.086411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992a7a4-1c85-4041-8708-0d94c6b29555 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.099217 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 406/816] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:50:31 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/ad2a56d3235b46e3b1fbb8bf0c2b4e0b/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:50:32.100019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992a7a4-1c85-4041-8708-0d94c6b29555 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-80c33d45-6de7-4374-977f-82bb8cb4a789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:32.101476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:32.102536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:32.104110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:32.104110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:32.104110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:32.104110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:32.104110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 411/817] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:32.104110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c33d45-6de7-4374-977f-82bb8cb4a789 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.225230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c33d45-6de7-4374-977f-82bb8cb4a789 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.229636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992a7a4-1c85-4041-8708-0d94c6b29555 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.269839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:32.275725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364acf2e-6bf7-4e53-be98-bf08d68ddeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.276674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364acf2e-6bf7-4e53-be98-bf08d68ddeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.276799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b8682-22ed-4cc5-a024-301d939ecdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.277301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364acf2e-6bf7-4e53-be98-bf08d68ddeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.277550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b8682-22ed-4cc5-a024-301d939ecdae None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.277835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b8682-22ed-4cc5-a024-301d939ecdae None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.278115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b8682-22ed-4cc5-a024-301d939ecdae None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.278379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364acf2e-6bf7-4e53-be98-bf08d68ddeb9 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.278827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b8682-22ed-4cc5-a024-301d939ecdae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:32.279137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b8682-22ed-4cc5-a024-301d939ecdae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.279710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364acf2e-6bf7-4e53-be98-bf08d68ddeb9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:32.280197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364acf2e-6bf7-4e53-be98-bf08d68ddeb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.280500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b8682-22ed-4cc5-a024-301d939ecdae None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.281963 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364acf2e-6bf7-4e53-be98-bf08d68ddeb9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.285331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 412/818] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:32.286109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b8682-22ed-4cc5-a024-301d939ecdae None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.287538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 407/819] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:32.288612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364acf2e-6bf7-4e53-be98-bf08d68ddeb9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.348620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:32.404754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b8682-22ed-4cc5-a024-301d939ecdae None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.412364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364acf2e-6bf7-4e53-be98-bf08d68ddeb9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.439126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d6672-02dc-42cb-88fb-8027299ec96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.440012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d6672-02dc-42cb-88fb-8027299ec96f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.440306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d6672-02dc-42cb-88fb-8027299ec96f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.440587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d6672-02dc-42cb-88fb-8027299ec96f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.451041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdbb11-8aa2-4a7a-906d-e1ef9d85cb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.451709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdbb11-8aa2-4a7a-906d-e1ef9d85cb54 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.451997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdbb11-8aa2-4a7a-906d-e1ef9d85cb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.452275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdbb11-8aa2-4a7a-906d-e1ef9d85cb54 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.452388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d6672-02dc-42cb-88fb-8027299ec96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.452998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdbb11-8aa2-4a7a-906d-e1ef9d85cb54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:32.453346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdbb11-8aa2-4a7a-906d-e1ef9d85cb54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.453761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d6672-02dc-42cb-88fb-8027299ec96f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.454701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdbb11-8aa2-4a7a-906d-e1ef9d85cb54 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.471378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 408/820] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:32.471378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdbb11-8aa2-4a7a-906d-e1ef9d85cb54 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-ce2d6672-02dc-42cb-88fb-8027299ec96f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:32.471932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:32.472799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:32.473730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:32.473730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:32.473730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:32.473730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:32.473730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 413/821] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:32.473730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d6672-02dc-42cb-88fb-8027299ec96f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.530117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:32.579236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdbb11-8aa2-4a7a-906d-e1ef9d85cb54 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.581609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:32.587624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d6672-02dc-42cb-88fb-8027299ec96f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.616422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0d533-516d-453b-97fb-102b02f6afa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.617013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0d533-516d-453b-97fb-102b02f6afa8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.617339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0d533-516d-453b-97fb-102b02f6afa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.617624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0d533-516d-453b-97fb-102b02f6afa8 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.618343 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0d533-516d-453b-97fb-102b02f6afa8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:32.618661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0d533-516d-453b-97fb-102b02f6afa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.620591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0d533-516d-453b-97fb-102b02f6afa8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.623178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27431ec1-51b1-487a-b4dc-82dffdc36673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.624017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27431ec1-51b1-487a-b4dc-82dffdc36673 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.624444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27431ec1-51b1-487a-b4dc-82dffdc36673 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.624867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27431ec1-51b1-487a-b4dc-82dffdc36673 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.625635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 409/822] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:32.625913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27431ec1-51b1-487a-b4dc-82dffdc36673 None admin] RBAC: Adding query filter params (type=network) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:32.626349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27431ec1-51b1-487a-b4dc-82dffdc36673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.626447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0d533-516d-453b-97fb-102b02f6afa8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.628008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27431ec1-51b1-487a-b4dc-82dffdc36673 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.635497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 414/823] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:32.640740 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27431ec1-51b1-487a-b4dc-82dffdc36673 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.731594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0d533-516d-453b-97fb-102b02f6afa8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.741959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27431ec1-51b1-487a-b4dc-82dffdc36673 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.755322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:32.768706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcf5db-ada1-42c8-aa1a-1fadd99595a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.769536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcf5db-ada1-42c8-aa1a-1fadd99595a0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.769825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcf5db-ada1-42c8-aa1a-1fadd99595a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.770108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcf5db-ada1-42c8-aa1a-1fadd99595a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.777964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dcf5db-ada1-42c8-aa1a-1fadd99595a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.778726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd593bc7-9bc6-4b26-b732-d8eb7522e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.779433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dcf5db-ada1-42c8-aa1a-1fadd99595a0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.779844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd593bc7-9bc6-4b26-b732-d8eb7522e748 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.780271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd593bc7-9bc6-4b26-b732-d8eb7522e748 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.780733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd593bc7-9bc6-4b26-b732-d8eb7522e748 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.781631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd593bc7-9bc6-4b26-b732-d8eb7522e748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:32.782083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd593bc7-9bc6-4b26-b732-d8eb7522e748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.783588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd593bc7-9bc6-4b26-b732-d8eb7522e748 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-06dcf5db-ada1-42c8-aa1a-1fadd99595a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:32.793720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:32.794782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:32.795729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:32.795729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:32.795729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:32.795729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:32.795729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:32.795729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:32.795729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:32.795729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:32.795729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 410/824] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:32.796214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcf5db-ada1-42c8-aa1a-1fadd99595a0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.801831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 415/825] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:32.802857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd593bc7-9bc6-4b26-b732-d8eb7522e748 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.827348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:32.903032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcf5db-ada1-42c8-aa1a-1fadd99595a0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.913266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd593bc7-9bc6-4b26-b732-d8eb7522e748 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:32.944682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf2149-914b-491a-89c5-08ff27741861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.945754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf2149-914b-491a-89c5-08ff27741861 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.946311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf2149-914b-491a-89c5-08ff27741861 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.946657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf2149-914b-491a-89c5-08ff27741861 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.947476 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf2149-914b-491a-89c5-08ff27741861 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.949028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf2149-914b-491a-89c5-08ff27741861 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.950029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20897374-85c5-4cf8-bd66-71e90b906198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:32.950686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20897374-85c5-4cf8-bd66-71e90b906198 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:32.950976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20897374-85c5-4cf8-bd66-71e90b906198 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:32.951261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20897374-85c5-4cf8-bd66-71e90b906198 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:32.952005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20897374-85c5-4cf8-bd66-71e90b906198 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:32.953404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20897374-85c5-4cf8-bd66-71e90b906198 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:32.971731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 411/826] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jun 21 18:50:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:32.972898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf2149-914b-491a-89c5-08ff27741861 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:32.983450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 416/827] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jun 21 18:50:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:32.983450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20897374-85c5-4cf8-bd66-71e90b906198 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:33.023060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:33.158544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20897374-85c5-4cf8-bd66-71e90b906198 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:33.208332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:33.222234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef021-4d8a-4cca-a791-7f2f70f7deff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:33.223186 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef021-4d8a-4cca-a791-7f2f70f7deff None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:33.223581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef021-4d8a-4cca-a791-7f2f70f7deff None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:33.223986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef021-4d8a-4cca-a791-7f2f70f7deff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:33.234469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeef021-4d8a-4cca-a791-7f2f70f7deff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:33.236110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeef021-4d8a-4cca-a791-7f2f70f7deff None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:33.237386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf2149-914b-491a-89c5-08ff27741861 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-1aeef021-4d8a-4cca-a791-7f2f70f7deff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:33.253805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:33.254960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:33.256079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:33.256079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:33.256079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:33.256079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:33.256079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:33.256079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:33.256079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:33.256079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:33.257179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 417/828] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:33.262744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef021-4d8a-4cca-a791-7f2f70f7deff None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:33.281059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a056617-4b83-4f8b-849d-bf94f2162771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:33.282062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a056617-4b83-4f8b-849d-bf94f2162771 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:33.282529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a056617-4b83-4f8b-849d-bf94f2162771 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:33.282989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a056617-4b83-4f8b-849d-bf94f2162771 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:33.284091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a056617-4b83-4f8b-849d-bf94f2162771 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:33.284688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a056617-4b83-4f8b-849d-bf94f2162771 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:33.286444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a056617-4b83-4f8b-849d-bf94f2162771 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:33.300591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 412/829] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:33.307079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a056617-4b83-4f8b-849d-bf94f2162771 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:33.363403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:33.380993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef021-4d8a-4cca-a791-7f2f70f7deff None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:33.423000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec90cf2-a66c-4d05-88ec-79e1d7a134fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:33.423570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec90cf2-a66c-4d05-88ec-79e1d7a134fa None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:33.423858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec90cf2-a66c-4d05-88ec-79e1d7a134fa None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:33.424135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec90cf2-a66c-4d05-88ec-79e1d7a134fa None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:33.424984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec90cf2-a66c-4d05-88ec-79e1d7a134fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:33.425351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec90cf2-a66c-4d05-88ec-79e1d7a134fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:33.426840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec90cf2-a66c-4d05-88ec-79e1d7a134fa None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:33.438962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a056617-4b83-4f8b-849d-bf94f2162771 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:33.440299 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 418/830] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:33.447127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec90cf2-a66c-4d05-88ec-79e1d7a134fa None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:33.476732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7a837-a5e3-447c-ad57-7cece6c04bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:33.477980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7a837-a5e3-447c-ad57-7cece6c04bc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:33.478563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7a837-a5e3-447c-ad57-7cece6c04bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:33.479055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7a837-a5e3-447c-ad57-7cece6c04bc9 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/9fee9b6e04f24f96adb1a70306463a91/roles/37aa79fddde643aeab12191e2c79681f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:33.490902 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:33.496532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7a837-a5e3-447c-ad57-7cece6c04bc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=9fee9b6e04f24f96adb1a70306463a91, role_id=37aa79fddde643aeab12191e2c79681f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:33.498384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7a837-a5e3-447c-ad57-7cece6c04bc9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:33.513223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 413/831] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:50:33 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/9fee9b6e04f24f96adb1a70306463a91/roles/37aa79fddde643aeab12191e2c79681f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:50:33.563767 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec90cf2-a66c-4d05-88ec-79e1d7a134fa None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:33.612332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b409a7b-aa60-42d9-83ed-ff57f7c0b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:33.613006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b409a7b-aa60-42d9-83ed-ff57f7c0b078 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:33.613328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b409a7b-aa60-42d9-83ed-ff57f7c0b078 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:33.613610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b409a7b-aa60-42d9-83ed-ff57f7c0b078 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:33.614460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b409a7b-aa60-42d9-83ed-ff57f7c0b078 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=dcbcec65220c4ebb95f974925fa73f34) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:33.614777 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b409a7b-aa60-42d9-83ed-ff57f7c0b078 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:33.616239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b409a7b-aa60-42d9-83ed-ff57f7c0b078 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:33.624186 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 419/832] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:33 2022] GET /identity/v3/role_assignments?user.id=dcbcec65220c4ebb95f974925fa73f34&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=e089a0d6040040c4bcf143a680488e3d => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:34.172785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535424b-916f-4766-a6a7-dd8d526782d1 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:34.173369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535424b-916f-4766-a6a7-dd8d526782d1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:34.173638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535424b-916f-4766-a6a7-dd8d526782d1 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:34.175043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 414/833] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:34.180480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e7dd1-c32b-45cb-84e2-35b8f41298cb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:34.180901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e7dd1-c32b-45cb-84e2-35b8f41298cb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:34.181118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e7dd1-c32b-45cb-84e2-35b8f41298cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:34.198272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e51e7dd1-c32b-45cb-84e2-35b8f41298cb None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:34.199504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e51e7dd1-c32b-45cb-84e2-35b8f41298cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:34.201489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:34.272851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 420/834] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:34.297559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535424b-916f-4766-a6a7-dd8d526782d1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:34.414049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535424b-916f-4766-a6a7-dd8d526782d1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:34.449615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a3c80-269a-4d1f-8dc1-b125b7759a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:34.450631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3c80-269a-4d1f-8dc1-b125b7759a26 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:34.451061 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3c80-269a-4d1f-8dc1-b125b7759a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:34.451506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3c80-269a-4d1f-8dc1-b125b7759a26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:34.462105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a3c80-269a-4d1f-8dc1-b125b7759a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-877a3c80-269a-4d1f-8dc1-b125b7759a26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:34.470610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:34.471767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:50:34.472836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:34.479786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 415/835] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:34.481420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e7dd1-c32b-45cb-84e2-35b8f41298cb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:34.580349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e7dd1-c32b-45cb-84e2-35b8f41298cb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:34.614808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe091c-c12f-488a-83a5-1b464ca456d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:34.615689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe091c-c12f-488a-83a5-1b464ca456d4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:34.616123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe091c-c12f-488a-83a5-1b464ca456d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:34.616570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe091c-c12f-488a-83a5-1b464ca456d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:34.617479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe091c-c12f-488a-83a5-1b464ca456d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:34.617931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe091c-c12f-488a-83a5-1b464ca456d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:34.619449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe091c-c12f-488a-83a5-1b464ca456d4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:34.624783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:34.632609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 421/836] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:34.633949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a3c80-269a-4d1f-8dc1-b125b7759a26 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:34.702116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:34.748599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a3c80-269a-4d1f-8dc1-b125b7759a26 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:34.783705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c319ac1-e228-4113-b51d-b1b5ca423b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:34.784497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c319ac1-e228-4113-b51d-b1b5ca423b0a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:34.784780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c319ac1-e228-4113-b51d-b1b5ca423b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:34.785054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c319ac1-e228-4113-b51d-b1b5ca423b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:34.796124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c319ac1-e228-4113-b51d-b1b5ca423b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:34.797728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c319ac1-e228-4113-b51d-b1b5ca423b0a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2c319ac1-e228-4113-b51d-b1b5ca423b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:34.816000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:34.816990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:34.817917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:34.817917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:34.817917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:34.817917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:34.821250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 416/837] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:34.823870 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe091c-c12f-488a-83a5-1b464ca456d4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:34.927712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe091c-c12f-488a-83a5-1b464ca456d4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:34.963697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc4612-805b-436a-9bb8-33ce691a6479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:34.964580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc4612-805b-436a-9bb8-33ce691a6479 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:34.965015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc4612-805b-436a-9bb8-33ce691a6479 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:34.965465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc4612-805b-436a-9bb8-33ce691a6479 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:34.966330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc4612-805b-436a-9bb8-33ce691a6479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:34.966808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc4612-805b-436a-9bb8-33ce691a6479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:34.968311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc4612-805b-436a-9bb8-33ce691a6479 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:34.981828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 422/838] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:34.987530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c319ac1-e228-4113-b51d-b1b5ca423b0a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:35.090884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c319ac1-e228-4113-b51d-b1b5ca423b0a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:35.092536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:35.129977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92045cf4-3095-422a-803e-b15a8cba6704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:35.130772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92045cf4-3095-422a-803e-b15a8cba6704 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.131055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92045cf4-3095-422a-803e-b15a8cba6704 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.131330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92045cf4-3095-422a-803e-b15a8cba6704 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.139839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92045cf4-3095-422a-803e-b15a8cba6704 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:35.141302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92045cf4-3095-422a-803e-b15a8cba6704 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-92045cf4-3095-422a-803e-b15a8cba6704 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:35.155576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:35.156527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:35.157485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 417/839] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:50:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:35.162553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc4612-805b-436a-9bb8-33ce691a6479 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:35.226915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:35.273571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc4612-805b-436a-9bb8-33ce691a6479 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:35.309856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c47a1-a08a-4f96-9e15-a033c1a7ff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:35.310859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c47a1-a08a-4f96-9e15-a033c1a7ff5a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.311292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c47a1-a08a-4f96-9e15-a033c1a7ff5a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.311751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c47a1-a08a-4f96-9e15-a033c1a7ff5a None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.312995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c47a1-a08a-4f96-9e15-a033c1a7ff5a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:35.313469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c47a1-a08a-4f96-9e15-a033c1a7ff5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:35.315027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c47a1-a08a-4f96-9e15-a033c1a7ff5a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:35.333868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 423/840] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jun 21 18:50:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:35.339242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92045cf4-3095-422a-803e-b15a8cba6704 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:35.393444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80158cfc-2783-41fc-9e52-6043788bb1ce None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.393814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80158cfc-2783-41fc-9e52-6043788bb1ce None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.393999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80158cfc-2783-41fc-9e52-6043788bb1ce None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.395624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 424/841] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:35.402289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4155617-1e79-4f9c-b8d6-f49d0bc48570 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.402576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4155617-1e79-4f9c-b8d6-f49d0bc48570 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.402853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4155617-1e79-4f9c-b8d6-f49d0bc48570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.420728 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d4155617-1e79-4f9c-b8d6-f49d0bc48570 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:35.421948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d4155617-1e79-4f9c-b8d6-f49d0bc48570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:35.456255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92045cf4-3095-422a-803e-b15a8cba6704 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:35.491539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5c033-d203-4036-95ff-34a4c9bd4f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:35.492113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5c033-d203-4036-95ff-34a4c9bd4f82 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.492395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5c033-d203-4036-95ff-34a4c9bd4f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.492671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5c033-d203-4036-95ff-34a4c9bd4f82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.495293 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 425/842] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:35.518086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4155617-1e79-4f9c-b8d6-f49d0bc48570 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:35.527911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:35.577866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5c033-d203-4036-95ff-34a4c9bd4f82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:35.579372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5c033-d203-4036-95ff-34a4c9bd4f82 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:35.586711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:35.645979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4155617-1e79-4f9c-b8d6-f49d0bc48570 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:35.681898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20263b-b397-4ce4-b9f3-fe38ccfee169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:35.682723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20263b-b397-4ce4-b9f3-fe38ccfee169 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.683005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20263b-b397-4ce4-b9f3-fe38ccfee169 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.683279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20263b-b397-4ce4-b9f3-fe38ccfee169 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.683932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20263b-b397-4ce4-b9f3-fe38ccfee169 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:35.685325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20263b-b397-4ce4-b9f3-fe38ccfee169 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:35.695412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 418/843] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:35 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:50:35.697034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59312e-f5eb-4231-bb32-5541df493156 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-ac20263b-b397-4ce4-b9f3-fe38ccfee169 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:35.697520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:35.698469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:50:35.699318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:35.700645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59312e-f5eb-4231-bb32-5541df493156 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.700956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59312e-f5eb-4231-bb32-5541df493156 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.702579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 419/844] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:35.703584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 426/845] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:50:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:35.704667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20263b-b397-4ce4-b9f3-fe38ccfee169 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:35.705882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59312e-f5eb-4231-bb32-5541df493156 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:35.781493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:35.818735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20263b-b397-4ce4-b9f3-fe38ccfee169 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:35.830711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59312e-f5eb-4231-bb32-5541df493156 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:35.832394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:35.856431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4da7c-a36a-433f-ae1c-2606827d56b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:35.857172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4da7c-a36a-433f-ae1c-2606827d56b7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.857408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4da7c-a36a-433f-ae1c-2606827d56b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.857749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4da7c-a36a-433f-ae1c-2606827d56b7 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.858385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4da7c-a36a-433f-ae1c-2606827d56b7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:35.858696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4da7c-a36a-433f-ae1c-2606827d56b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:35.860055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4da7c-a36a-433f-ae1c-2606827d56b7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:35.869161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 427/846] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:50:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:35.869161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312e4d5-592a-45e8-a51a-c60f9d111071 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.869161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312e4d5-592a-45e8-a51a-c60f9d111071 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.869161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312e4d5-592a-45e8-a51a-c60f9d111071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.874084 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c2511-e027-432d-9e15-da3f2ef9a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:35.875028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c2511-e027-432d-9e15-da3f2ef9a087 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:35.875318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c2511-e027-432d-9e15-da3f2ef9a087 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:35.875599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c2511-e027-432d-9e15-da3f2ef9a087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:35.886118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5312e4d5-592a-45e8-a51a-c60f9d111071 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:35.887324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5312e4d5-592a-45e8-a51a-c60f9d111071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:35.958780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 428/847] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:35.959640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312e4d5-592a-45e8-a51a-c60f9d111071 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.003727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c2511-e027-432d-9e15-da3f2ef9a087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.005220 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c2511-e027-432d-9e15-da3f2ef9a087 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5d9c2511-e027-432d-9e15-da3f2ef9a087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:36.022844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:36.023917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:36.025610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:36.025610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:36.025610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:36.025610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:36.025610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 420/848] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:36.026043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420d3d6-93c4-42a1-b827-23444ed3e771 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.026162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420d3d6-93c4-42a1-b827-23444ed3e771 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.026445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420d3d6-93c4-42a1-b827-23444ed3e771 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.027856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 421/849] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:36.032358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420d3d6-93c4-42a1-b827-23444ed3e771 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.064083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312e4d5-592a-45e8-a51a-c60f9d111071 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.071450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:36.101664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70628af-8943-4bcb-93a8-05f15d015727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.102329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70628af-8943-4bcb-93a8-05f15d015727 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.102607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70628af-8943-4bcb-93a8-05f15d015727 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.102885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70628af-8943-4bcb-93a8-05f15d015727 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.103643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70628af-8943-4bcb-93a8-05f15d015727 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:36.103954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70628af-8943-4bcb-93a8-05f15d015727 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.105350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70628af-8943-4bcb-93a8-05f15d015727 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.109773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 429/850] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:50:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:36.110552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70628af-8943-4bcb-93a8-05f15d015727 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.139214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420d3d6-93c4-42a1-b827-23444ed3e771 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.158244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:36.184463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28a233-f86d-431f-a3ce-c083c88d22c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.185298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28a233-f86d-431f-a3ce-c083c88d22c2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.185581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28a233-f86d-431f-a3ce-c083c88d22c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.185862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28a233-f86d-431f-a3ce-c083c88d22c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.199198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28a233-f86d-431f-a3ce-c083c88d22c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-da28a233-f86d-431f-a3ce-c083c88d22c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:36.207073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:36.208130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:36.208996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 422/851] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:36.210272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ebb5c-7bb4-488b-be70-f4dee9712d3c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.210377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ebb5c-7bb4-488b-be70-f4dee9712d3c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.210614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ebb5c-7bb4-488b-be70-f4dee9712d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.229516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e80ebb5c-7bb4-488b-be70-f4dee9712d3c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:36.230766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e80ebb5c-7bb4-488b-be70-f4dee9712d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:36.243870 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70628af-8943-4bcb-93a8-05f15d015727 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.284148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd7e9c-a8f4-47c8-a691-d984e04e9e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.284748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd7e9c-a8f4-47c8-a691-d984e04e9e24 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.285036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd7e9c-a8f4-47c8-a691-d984e04e9e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.285393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd7e9c-a8f4-47c8-a691-d984e04e9e24 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.286132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd7e9c-a8f4-47c8-a691-d984e04e9e24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:36.286442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd7e9c-a8f4-47c8-a691-d984e04e9e24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.287835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd7e9c-a8f4-47c8-a691-d984e04e9e24 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.300228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 430/852] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:36.301396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd7e9c-a8f4-47c8-a691-d984e04e9e24 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.316842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 423/853] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:36.322806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ebb5c-7bb4-488b-be70-f4dee9712d3c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.348294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:36.425927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd7e9c-a8f4-47c8-a691-d984e04e9e24 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.433007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ebb5c-7bb4-488b-be70-f4dee9712d3c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.436649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:36.462994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7b079-7b81-4cde-a13d-d7c1bd7b4182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.463667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7b079-7b81-4cde-a13d-d7c1bd7b4182 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.463961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7b079-7b81-4cde-a13d-d7c1bd7b4182 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.464248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7b079-7b81-4cde-a13d-d7c1bd7b4182 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.465000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7b079-7b81-4cde-a13d-d7c1bd7b4182 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:36.465346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7b079-7b81-4cde-a13d-d7c1bd7b4182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.466728 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7b079-7b81-4cde-a13d-d7c1bd7b4182 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.474777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c321ba-c00a-418e-a461-6f2022add05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.475200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 431/854] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:36.475938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c321ba-c00a-418e-a461-6f2022add05d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.476042 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7b079-7b81-4cde-a13d-d7c1bd7b4182 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.476553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c321ba-c00a-418e-a461-6f2022add05d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.476971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c321ba-c00a-418e-a461-6f2022add05d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.486113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c321ba-c00a-418e-a461-6f2022add05d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.487837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c321ba-c00a-418e-a461-6f2022add05d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-69c321ba-c00a-418e-a461-6f2022add05d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:36.503920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:36.505033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:36.506053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:36.506053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:36.506053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:36.506053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:36.506053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:36.506053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:36.506053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:36.506053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:36.506053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 424/855] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:36.506539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c321ba-c00a-418e-a461-6f2022add05d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.577502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:36.588157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7b079-7b81-4cde-a13d-d7c1bd7b4182 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.617367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c321ba-c00a-418e-a461-6f2022add05d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.626053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6560fcf-e596-4585-85ac-6c44620e55fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.626744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6560fcf-e596-4585-85ac-6c44620e55fa None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.627040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6560fcf-e596-4585-85ac-6c44620e55fa None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.627320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6560fcf-e596-4585-85ac-6c44620e55fa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.627977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6560fcf-e596-4585-85ac-6c44620e55fa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.629376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6560fcf-e596-4585-85ac-6c44620e55fa None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e6560fcf-e596-4585-85ac-6c44620e55fa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:36.642431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:36.643644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:36.644723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 432/856] 10.222.0.70 () {56 vars in 1132 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:36.645795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6560fcf-e596-4585-85ac-6c44620e55fa None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.660159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c13c3c-f58c-4168-9c7a-a983982b99a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.661363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c13c3c-f58c-4168-9c7a-a983982b99a3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.661657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c13c3c-f58c-4168-9c7a-a983982b99a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.662213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c13c3c-f58c-4168-9c7a-a983982b99a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.675694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c13c3c-f58c-4168-9c7a-a983982b99a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.677688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c13c3c-f58c-4168-9c7a-a983982b99a3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-93c13c3c-f58c-4168-9c7a-a983982b99a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:36.698190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:36.699126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:36.700096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:36.700096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:36.700096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:36.700096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:36.700096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 425/857] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:36.700383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c13c3c-f58c-4168-9c7a-a983982b99a3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.721313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:36.758951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6560fcf-e596-4585-85ac-6c44620e55fa None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.793076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac614019-e71f-4c5c-9f86-f8a0117727c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.793763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac614019-e71f-4c5c-9f86-f8a0117727c9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.794052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac614019-e71f-4c5c-9f86-f8a0117727c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.794334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac614019-e71f-4c5c-9f86-f8a0117727c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.795133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac614019-e71f-4c5c-9f86-f8a0117727c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:36.795447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac614019-e71f-4c5c-9f86-f8a0117727c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.796883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac614019-e71f-4c5c-9f86-f8a0117727c9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.810845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 433/858] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:36.811920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac614019-e71f-4c5c-9f86-f8a0117727c9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.821521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c13c3c-f58c-4168-9c7a-a983982b99a3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.832456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:36.856153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb6ccc-f63f-4f95-b75b-048707c5750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.856944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6ccc-f63f-4f95-b75b-048707c5750d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.857260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6ccc-f63f-4f95-b75b-048707c5750d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.857534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6ccc-f63f-4f95-b75b-048707c5750d None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.858241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb6ccc-f63f-4f95-b75b-048707c5750d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:36.858546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb6ccc-f63f-4f95-b75b-048707c5750d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.859923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb6ccc-f63f-4f95-b75b-048707c5750d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.870030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 426/859] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:36.870030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb6ccc-f63f-4f95-b75b-048707c5750d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.927657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac614019-e71f-4c5c-9f86-f8a0117727c9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:36.965569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:36.965760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527915a9-88c3-4163-97de-ee2025f94e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:36.966385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527915a9-88c3-4163-97de-ee2025f94e9d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:36.966666 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527915a9-88c3-4163-97de-ee2025f94e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:36.966943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527915a9-88c3-4163-97de-ee2025f94e9d None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:36.967668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527915a9-88c3-4163-97de-ee2025f94e9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:36.968315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527915a9-88c3-4163-97de-ee2025f94e9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:36.969662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527915a9-88c3-4163-97de-ee2025f94e9d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:36.981641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 434/860] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:36.982458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527915a9-88c3-4163-97de-ee2025f94e9d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:36.994341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb6ccc-f63f-4f95-b75b-048707c5750d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:37.030059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9382d-05af-49e0-bdf0-2550e7c98a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:37.031003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9382d-05af-49e0-bdf0-2550e7c98a47 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:37.031314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9382d-05af-49e0-bdf0-2550e7c98a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:37.031597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9382d-05af-49e0-bdf0-2550e7c98a47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:37.032944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9382d-05af-49e0-bdf0-2550e7c98a47 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:37.033300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9382d-05af-49e0-bdf0-2550e7c98a47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:37.034770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9382d-05af-49e0-bdf0-2550e7c98a47 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:37.041954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 427/861] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/role_assignments?user.id=ad2a56d3235b46e3b1fbb8bf0c2b4e0b&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=e089a0d6040040c4bcf143a680488e3d => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:37.050939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9382d-05af-49e0-bdf0-2550e7c98a47 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:37.068736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:37.101792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527915a9-88c3-4163-97de-ee2025f94e9d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:37.143880 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344e2aa-b9ba-4633-a765-60959485679a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:37.144658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344e2aa-b9ba-4633-a765-60959485679a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:37.145010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344e2aa-b9ba-4633-a765-60959485679a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:37.145389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344e2aa-b9ba-4633-a765-60959485679a None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:37.146497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344e2aa-b9ba-4633-a765-60959485679a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:37.146638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344e2aa-b9ba-4633-a765-60959485679a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:37.148162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344e2aa-b9ba-4633-a765-60959485679a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:37.154598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 435/862] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:37.205596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9382d-05af-49e0-bdf0-2550e7c98a47 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:37.249708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba91f9-e88a-47ac-8cff-b701fdc4b69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:37.250875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba91f9-e88a-47ac-8cff-b701fdc4b69f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:37.251335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba91f9-e88a-47ac-8cff-b701fdc4b69f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:37.251772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba91f9-e88a-47ac-8cff-b701fdc4b69f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:37.261327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba91f9-e88a-47ac-8cff-b701fdc4b69f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:37.263068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba91f9-e88a-47ac-8cff-b701fdc4b69f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c1ba91f9-e88a-47ac-8cff-b701fdc4b69f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:37.280231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:37.281473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:37.282668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:37.284866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 428/863] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:50:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:37.291477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344e2aa-b9ba-4633-a765-60959485679a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:37.420668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344e2aa-b9ba-4633-a765-60959485679a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:37.456886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77a3f5-c27e-45e3-98ed-36cba8c6bc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:37.457964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77a3f5-c27e-45e3-98ed-36cba8c6bc1f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:37.458457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77a3f5-c27e-45e3-98ed-36cba8c6bc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:37.458914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77a3f5-c27e-45e3-98ed-36cba8c6bc1f None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:37.459938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77a3f5-c27e-45e3-98ed-36cba8c6bc1f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:37.460452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77a3f5-c27e-45e3-98ed-36cba8c6bc1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:37.462096 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77a3f5-c27e-45e3-98ed-36cba8c6bc1f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:37.473006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:37.483570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 436/864] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jun 21 18:50:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:37.488944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba91f9-e88a-47ac-8cff-b701fdc4b69f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:37.492862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:37.597082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba91f9-e88a-47ac-8cff-b701fdc4b69f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:37.635144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a93c4a-852b-4861-a4ce-b5c557a09b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:37.636039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a93c4a-852b-4861-a4ce-b5c557a09b4d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:37.636463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a93c4a-852b-4861-a4ce-b5c557a09b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:37.636875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a93c4a-852b-4861-a4ce-b5c557a09b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:37.714958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a93c4a-852b-4861-a4ce-b5c557a09b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:37.716440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a93c4a-852b-4861-a4ce-b5c557a09b4d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:37.719619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:37.828862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 429/865] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:37 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:50:37.834433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77a3f5-c27e-45e3-98ed-36cba8c6bc1f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:37.947531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77a3f5-c27e-45e3-98ed-36cba8c6bc1f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:37.983370 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862e910-33fe-4fbd-aa9f-f8d0f9a6e6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:37.984074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862e910-33fe-4fbd-aa9f-f8d0f9a6e6ba None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:37.984415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862e910-33fe-4fbd-aa9f-f8d0f9a6e6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:37.984698 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862e910-33fe-4fbd-aa9f-f8d0f9a6e6ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:37.996760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862e910-33fe-4fbd-aa9f-f8d0f9a6e6ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:37.998227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862e910-33fe-4fbd-aa9f-f8d0f9a6e6ba None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-9862e910-33fe-4fbd-aa9f-f8d0f9a6e6ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:38.016741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:38.017734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:38.018633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:38.018633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:38.018633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:38.018633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:38.018633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 437/866] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:38.023987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a93c4a-852b-4861-a4ce-b5c557a09b4d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:38.131532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a93c4a-852b-4861-a4ce-b5c557a09b4d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:38.134005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:38.171102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375c1ea-4c45-4345-822c-eef8bac9ef5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:38.171777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375c1ea-4c45-4345-822c-eef8bac9ef5e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:38.172063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375c1ea-4c45-4345-822c-eef8bac9ef5e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:38.172344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375c1ea-4c45-4345-822c-eef8bac9ef5e None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:38.173075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375c1ea-4c45-4345-822c-eef8bac9ef5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:38.173413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375c1ea-4c45-4345-822c-eef8bac9ef5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:38.174851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375c1ea-4c45-4345-822c-eef8bac9ef5e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:38.176525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e131b47-0299-4276-81c3-842b167300ea None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:38.176811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e131b47-0299-4276-81c3-842b167300ea None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:38.177111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e131b47-0299-4276-81c3-842b167300ea None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:38.178645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 438/867] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:38.184015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178bc72-ba0d-4d44-a893-3241ac47c3ba None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:38.184585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178bc72-ba0d-4d44-a893-3241ac47c3ba None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:38.185079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178bc72-ba0d-4d44-a893-3241ac47c3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:38.188019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 430/868] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:38.193423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375c1ea-4c45-4345-822c-eef8bac9ef5e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:38.204144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9178bc72-ba0d-4d44-a893-3241ac47c3ba None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:38.205737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9178bc72-ba0d-4d44-a893-3241ac47c3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:38.255684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:38.285310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 439/869] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:38.302580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375c1ea-4c45-4345-822c-eef8bac9ef5e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:38.306745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178bc72-ba0d-4d44-a893-3241ac47c3ba None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:38.354049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723d09-5202-460a-820a-e134734d172c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:38.354844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723d09-5202-460a-820a-e134734d172c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:38.355127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723d09-5202-460a-820a-e134734d172c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:38.355397 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723d09-5202-460a-820a-e134734d172c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:38.366220 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32723d09-5202-460a-820a-e134734d172c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:38.367949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32723d09-5202-460a-820a-e134734d172c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-32723d09-5202-460a-820a-e134734d172c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:38.384874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:38.385877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:38.386945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:38.386945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:38.386945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:38.386945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:38.386945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:38.386945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:38.386945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:38.386945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:38.386945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 431/870] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:38.392665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723d09-5202-460a-820a-e134734d172c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:38.420989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:38.444606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178bc72-ba0d-4d44-a893-3241ac47c3ba None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:38.481626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db8f12-0a86-45a0-9677-580bbfa9531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:38.482290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db8f12-0a86-45a0-9677-580bbfa9531a None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:38.482578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db8f12-0a86-45a0-9677-580bbfa9531a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:38.482852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db8f12-0a86-45a0-9677-580bbfa9531a None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:38.483586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db8f12-0a86-45a0-9677-580bbfa9531a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:38.485010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db8f12-0a86-45a0-9677-580bbfa9531a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:38.500130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 440/871] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:50:38 2022] POST /identity/v3/services => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:38.504419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723d09-5202-460a-820a-e134734d172c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:38.552809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7b613-2b87-42d2-b2b0-fff4544bceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:38.553438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b613-2b87-42d2-b2b0-fff4544bceb8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:38.553741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b613-2b87-42d2-b2b0-fff4544bceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:38.554010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b613-2b87-42d2-b2b0-fff4544bceb8 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:38.554840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7b613-2b87-42d2-b2b0-fff4544bceb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:38.555158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7b613-2b87-42d2-b2b0-fff4544bceb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:38.557305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7b613-2b87-42d2-b2b0-fff4544bceb8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:38.571525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 432/872] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:38.577605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db8f12-0a86-45a0-9677-580bbfa9531a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:38.630813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:38.701075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db8f12-0a86-45a0-9677-580bbfa9531a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:38.740662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e69021-e002-4bc1-b91e-313141ebb445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:38.741830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e69021-e002-4bc1-b91e-313141ebb445 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:38.742348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e69021-e002-4bc1-b91e-313141ebb445 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:38.742821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e69021-e002-4bc1-b91e-313141ebb445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:38.743886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e69021-e002-4bc1-b91e-313141ebb445 None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=9fee9b6e04f24f96adb1a70306463a91) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:38.744399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e69021-e002-4bc1-b91e-313141ebb445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:38.746079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e69021-e002-4bc1-b91e-313141ebb445 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:38.753196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 441/873] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:38 2022] GET /identity/v3/role_assignments?user.id=9fee9b6e04f24f96adb1a70306463a91&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:39.334034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f332b2b-2eab-4549-8986-0a4e6e4fa828 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:39.334671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f332b2b-2eab-4549-8986-0a4e6e4fa828 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:39.335197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f332b2b-2eab-4549-8986-0a4e6e4fa828 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:39.337073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 433/874] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:39.342734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309705a1-074b-434c-a318-0452d32de8a8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:39.343611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309705a1-074b-434c-a318-0452d32de8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:39.344088 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309705a1-074b-434c-a318-0452d32de8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:39.361424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-309705a1-074b-434c-a318-0452d32de8a8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:39.362937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-309705a1-074b-434c-a318-0452d32de8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:39.379411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d19a1-2fd7-437d-a473-cc661747f9eb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:39.379727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d19a1-2fd7-437d-a473-cc661747f9eb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:39.379974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d19a1-2fd7-437d-a473-cc661747f9eb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:39.381632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 434/875] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:39.388179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a01fb-1eae-40b5-afc0-fd5ca30d5cf6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:39.388745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a01fb-1eae-40b5-afc0-fd5ca30d5cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:39.389313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a01fb-1eae-40b5-afc0-fd5ca30d5cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:39.406586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1d4a01fb-1eae-40b5-afc0-fd5ca30d5cf6 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:39.408097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1d4a01fb-1eae-40b5-afc0-fd5ca30d5cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:39.452720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 442/876] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:39.476491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309705a1-074b-434c-a318-0452d32de8a8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:39.481042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 435/877] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:39.504728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a01fb-1eae-40b5-afc0-fd5ca30d5cf6 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:39.516011 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:39.518724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:39.596727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309705a1-074b-434c-a318-0452d32de8a8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:39.612066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a01fb-1eae-40b5-afc0-fd5ca30d5cf6 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:39.638237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b64b-bff5-4ef2-a886-dcf711dd5335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:39.639523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b64b-bff5-4ef2-a886-dcf711dd5335 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:39.640018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b64b-bff5-4ef2-a886-dcf711dd5335 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:39.640529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b64b-bff5-4ef2-a886-dcf711dd5335 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:39.641546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8b64b-bff5-4ef2-a886-dcf711dd5335 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:39.643273 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8b64b-bff5-4ef2-a886-dcf711dd5335 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:39.650082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a1555-537f-4c4b-ab15-15a9b32aad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:39.650768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a1555-537f-4c4b-ab15-15a9b32aad54 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:39.651051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a1555-537f-4c4b-ab15-15a9b32aad54 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:39.651328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a1555-537f-4c4b-ab15-15a9b32aad54 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:39.652090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a1555-537f-4c4b-ab15-15a9b32aad54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:39.653562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a1555-537f-4c4b-ab15-15a9b32aad54 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a0f8b64b-bff5-4ef2-a886-dcf711dd5335 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:39.660076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:39.661365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:39.662652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:39.671025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 443/878] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:39.679727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b64b-bff5-4ef2-a886-dcf711dd5335 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:39.689513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 436/879] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:50:39 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:39.804972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:39.810501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b64b-bff5-4ef2-a886-dcf711dd5335 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:39.855998 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfca10-ca39-4e4f-8270-0801e138c3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:39.856624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfca10-ca39-4e4f-8270-0801e138c3ca None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:39.856914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfca10-ca39-4e4f-8270-0801e138c3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:39.857220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfca10-ca39-4e4f-8270-0801e138c3ca None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:39.857934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfca10-ca39-4e4f-8270-0801e138c3ca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:39.858248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfca10-ca39-4e4f-8270-0801e138c3ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:39.859622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfca10-ca39-4e4f-8270-0801e138c3ca None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:39.865155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 444/880] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:39.870854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a1555-537f-4c4b-ab15-15a9b32aad54 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:39.947713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:39.985530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a1555-537f-4c4b-ab15-15a9b32aad54 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:40.024795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb2d26-bc95-4e0e-9da7-c35e5fe66628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:40.026016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb2d26-bc95-4e0e-9da7-c35e5fe66628 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:40.026303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb2d26-bc95-4e0e-9da7-c35e5fe66628 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:40.026588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb2d26-bc95-4e0e-9da7-c35e5fe66628 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:40.027313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb2d26-bc95-4e0e-9da7-c35e5fe66628 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:40.027634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb2d26-bc95-4e0e-9da7-c35e5fe66628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:40.029037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb2d26-bc95-4e0e-9da7-c35e5fe66628 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:40.034774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 437/881] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:40.631735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2eb52-5054-421a-90b4-295eef2fc351 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:40.632242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2eb52-5054-421a-90b4-295eef2fc351 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:40.632530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2eb52-5054-421a-90b4-295eef2fc351 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:40.634241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 445/882] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:40.640081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a269dc4-8daf-4df8-bff9-a0194d6a3919 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:40.640575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a269dc4-8daf-4df8-bff9-a0194d6a3919 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:40.640652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a269dc4-8daf-4df8-bff9-a0194d6a3919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:40.658291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7a269dc4-8daf-4df8-bff9-a0194d6a3919 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:40.659517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7a269dc4-8daf-4df8-bff9-a0194d6a3919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:40.746627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 438/883] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:40.769587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2eb52-5054-421a-90b4-295eef2fc351 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:40.882509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2eb52-5054-421a-90b4-295eef2fc351 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:40.922495 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a49da-6768-4821-afe4-a038ad23e1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:40.923311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a49da-6768-4821-afe4-a038ad23e1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:40.923603 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a49da-6768-4821-afe4-a038ad23e1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:40.923887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a49da-6768-4821-afe4-a038ad23e1d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:40.924602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a49da-6768-4821-afe4-a038ad23e1d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:40.926198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a49da-6768-4821-afe4-a038ad23e1d2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-661a49da-6768-4821-afe4-a038ad23e1d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:40.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:40.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:40.941018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 446/884] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:50:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:40.946839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a269dc4-8daf-4df8-bff9-a0194d6a3919 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:40.951326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d77eb-2401-44d1-a324-abad944e0109 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:40.951627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d77eb-2401-44d1-a324-abad944e0109 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:40.951911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d77eb-2401-44d1-a324-abad944e0109 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:40.953599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 447/885] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:40.959397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab7a6f-f6ef-4049-a14e-1827bf3c4250 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:40.959696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab7a6f-f6ef-4049-a14e-1827bf3c4250 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:40.960279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab7a6f-f6ef-4049-a14e-1827bf3c4250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:40.983231 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-dcab7a6f-f6ef-4049-a14e-1827bf3c4250 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:40.985651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dcab7a6f-f6ef-4049-a14e-1827bf3c4250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:41.012674 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:41.026666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:41.078184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a269dc4-8daf-4df8-bff9-a0194d6a3919 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:41.089634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 448/886] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:41.113332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab7a6f-f6ef-4049-a14e-1827bf3c4250 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:41.126858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d66c32-2ef3-48ea-a848-08ec8bb6bb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:41.128730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d66c32-2ef3-48ea-a848-08ec8bb6bb53 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.129645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d66c32-2ef3-48ea-a848-08ec8bb6bb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.130164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d66c32-2ef3-48ea-a848-08ec8bb6bb53 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.132089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d66c32-2ef3-48ea-a848-08ec8bb6bb53 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:41.133267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d66c32-2ef3-48ea-a848-08ec8bb6bb53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:41.136725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d66c32-2ef3-48ea-a848-08ec8bb6bb53 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:41.151080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 439/887] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:50:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:41.158720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d66c32-2ef3-48ea-a848-08ec8bb6bb53 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:41.229621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab7a6f-f6ef-4049-a14e-1827bf3c4250 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:41.270360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5821553-2b01-4d1c-a6f3-956a400c015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:41.271706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5821553-2b01-4d1c-a6f3-956a400c015d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.271996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5821553-2b01-4d1c-a6f3-956a400c015d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.272280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5821553-2b01-4d1c-a6f3-956a400c015d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.285352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5821553-2b01-4d1c-a6f3-956a400c015d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:41.287780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d66c32-2ef3-48ea-a848-08ec8bb6bb53 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a5821553-2b01-4d1c-a6f3-956a400c015d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:41.294014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:41.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:41.296102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 449/888] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jun 21 18:50:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:41.301717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5821553-2b01-4d1c-a6f3-956a400c015d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:41.330146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:41.332192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:41.343840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8032b2-1cd8-44c2-852b-b7e5a9b5413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:41.345185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8032b2-1cd8-44c2-852b-b7e5a9b5413b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.346124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8032b2-1cd8-44c2-852b-b7e5a9b5413b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.346634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8032b2-1cd8-44c2-852b-b7e5a9b5413b None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.347832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8032b2-1cd8-44c2-852b-b7e5a9b5413b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:41.348696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8032b2-1cd8-44c2-852b-b7e5a9b5413b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:41.351034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8032b2-1cd8-44c2-852b-b7e5a9b5413b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:41.367904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 440/889] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:50:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:41.369110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8032b2-1cd8-44c2-852b-b7e5a9b5413b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:41.453688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5821553-2b01-4d1c-a6f3-956a400c015d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:41.491923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224f136-99db-4aeb-903d-089d7cd3fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:41.492600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224f136-99db-4aeb-903d-089d7cd3fad9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.492884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224f136-99db-4aeb-903d-089d7cd3fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.493192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224f136-99db-4aeb-903d-089d7cd3fad9 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.494063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224f136-99db-4aeb-903d-089d7cd3fad9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:41.494290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224f136-99db-4aeb-903d-089d7cd3fad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:41.495685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224f136-99db-4aeb-903d-089d7cd3fad9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:41.505695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 450/890] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jun 21 18:50:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:41.510040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8032b2-1cd8-44c2-852b-b7e5a9b5413b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:41.511640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224f136-99db-4aeb-903d-089d7cd3fad9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:41.560451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8f7ff-82ee-40d0-9450-f7e20500c903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:41.562009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8f7ff-82ee-40d0-9450-f7e20500c903 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.563009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8f7ff-82ee-40d0-9450-f7e20500c903 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.563506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8f7ff-82ee-40d0-9450-f7e20500c903 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.564731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8f7ff-82ee-40d0-9450-f7e20500c903 None admin] RBAC: Adding query filter params (interface=public, service_id=64bf0d15d7fa4585979443ee4064803d, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:41.565297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8f7ff-82ee-40d0-9450-f7e20500c903 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:41.567682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8f7ff-82ee-40d0-9450-f7e20500c903 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:41.580331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 441/891] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jun 21 18:50:41 2022] GET /identity/v3/endpoints?service_id=64bf0d15d7fa4585979443ee4064803d&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:41.581471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8f7ff-82ee-40d0-9450-f7e20500c903 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:41.611260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:41.650001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:41.653104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224f136-99db-4aeb-903d-089d7cd3fad9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:41.698909 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8f7ff-82ee-40d0-9450-f7e20500c903 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:41.700459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6fd6d-1ca6-4f79-abc0-a4b448e7574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:41.702561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6fd6d-1ca6-4f79-abc0-a4b448e7574c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.703029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6fd6d-1ca6-4f79-abc0-a4b448e7574c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.703542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6fd6d-1ca6-4f79-abc0-a4b448e7574c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.724468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6fd6d-1ca6-4f79-abc0-a4b448e7574c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:41.726461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6fd6d-1ca6-4f79-abc0-a4b448e7574c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:41.739535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cacac-39be-4947-82dc-fa109b87f0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:41.740374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cacac-39be-4947-82dc-fa109b87f0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.740754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cacac-39be-4947-82dc-fa109b87f0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.741168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cacac-39be-4947-82dc-fa109b87f0e1 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.741957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cacac-39be-4947-82dc-fa109b87f0e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:41.744359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cacac-39be-4947-82dc-fa109b87f0e1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f7b6fd6d-1ca6-4f79-abc0-a4b448e7574c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:41.750325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:41.751413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:41.752558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:41.752558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:41.752558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:41.752558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:41.761123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 442/892] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jun 21 18:50:41 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:50:41.763458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 451/893] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:41.765149 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cacac-39be-4947-82dc-fa109b87f0e1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:41.866884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a5117-2fc2-4867-8660-45373dfb9185 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.867224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a5117-2fc2-4867-8660-45373dfb9185 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.867461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a5117-2fc2-4867-8660-45373dfb9185 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.869876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 452/894] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:41.875305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a3d5-1877-4a80-8c4f-f51222316415 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.875634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a3d5-1877-4a80-8c4f-f51222316415 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.876002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a3d5-1877-4a80-8c4f-f51222316415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.893840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cacac-39be-4947-82dc-fa109b87f0e1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:41.896081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a7b8a3d5-1877-4a80-8c4f-f51222316415 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:41.898315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a7b8a3d5-1877-4a80-8c4f-f51222316415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:41.926259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:41.945324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedb706-fbdf-4d68-9e70-9f2aeae69bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:41.946145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedb706-fbdf-4d68-9e70-9f2aeae69bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:41.946484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedb706-fbdf-4d68-9e70-9f2aeae69bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:41.946871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedb706-fbdf-4d68-9e70-9f2aeae69bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:41.947686 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedb706-fbdf-4d68-9e70-9f2aeae69bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:41.948062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedb706-fbdf-4d68-9e70-9f2aeae69bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:41.949816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedb706-fbdf-4d68-9e70-9f2aeae69bb4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:41.967980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 443/895] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:41.969337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedb706-fbdf-4d68-9e70-9f2aeae69bb4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:41.989018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 453/896] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:42.015792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a3d5-1877-4a80-8c4f-f51222316415 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:42.081841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:42.116494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedb706-fbdf-4d68-9e70-9f2aeae69bb4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:42.144963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a3d5-1877-4a80-8c4f-f51222316415 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:42.154986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ef840-4f53-4dd8-83f0-0b9ac86533cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:42.156164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef840-4f53-4dd8-83f0-0b9ac86533cf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.156750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef840-4f53-4dd8-83f0-0b9ac86533cf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.157260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef840-4f53-4dd8-83f0-0b9ac86533cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.166440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ef840-4f53-4dd8-83f0-0b9ac86533cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:42.168357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ef840-4f53-4dd8-83f0-0b9ac86533cf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:42.187729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a1878-0647-4f83-ad4f-f8e515dfebb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d08ef840-4f53-4dd8-83f0-0b9ac86533cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:42.188765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:42.189991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 444/897] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:50:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b42d-bdf7-4dd3-b904-1c7f8b9a0d4d None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.194346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b42d-bdf7-4dd3-b904-1c7f8b9a0d4d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.196567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b42d-bdf7-4dd3-b904-1c7f8b9a0d4d None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.196647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a1878-0647-4f83-ad4f-f8e515dfebb3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.196757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 445/898] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:42.196832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a1878-0647-4f83-ad4f-f8e515dfebb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.196832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a1878-0647-4f83-ad4f-f8e515dfebb3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.196832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a1878-0647-4f83-ad4f-f8e515dfebb3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:42.196832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a1878-0647-4f83-ad4f-f8e515dfebb3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:42.198729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6827d8-b39b-453c-ba9d-00ceff749488 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.199028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6827d8-b39b-453c-ba9d-00ceff749488 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.199572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6827d8-b39b-453c-ba9d-00ceff749488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-826a1878-0647-4f83-ad4f-f8e515dfebb3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:42.210626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:42.218268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:42.220509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 454/899] 10.222.0.70 () {56 vars in 1132 bytes} [Tue Jun 21 18:50:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:42.223429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a1878-0647-4f83-ad4f-f8e515dfebb3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:42.227501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8a6827d8-b39b-453c-ba9d-00ceff749488 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:42.229338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8a6827d8-b39b-453c-ba9d-00ceff749488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:42.268907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:42.305304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 446/900] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:42.306724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6827d8-b39b-453c-ba9d-00ceff749488 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:42.349454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a1878-0647-4f83-ad4f-f8e515dfebb3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:42.375750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:42.393453 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3033df-f909-457f-909f-41fb938366b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:42.394124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3033df-f909-457f-909f-41fb938366b1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.394406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3033df-f909-457f-909f-41fb938366b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.394687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3033df-f909-457f-909f-41fb938366b1 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.395671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3033df-f909-457f-909f-41fb938366b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:42.395983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3033df-f909-457f-909f-41fb938366b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:42.397587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3033df-f909-457f-909f-41fb938366b1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:42.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 455/901] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jun 21 18:50:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:42.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3033df-f909-457f-909f-41fb938366b1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:42.437840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6827d8-b39b-453c-ba9d-00ceff749488 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:42.480435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe80180-052d-43c6-88e6-b8811e5154b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:42.481475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe80180-052d-43c6-88e6-b8811e5154b5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.481936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe80180-052d-43c6-88e6-b8811e5154b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.482360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe80180-052d-43c6-88e6-b8811e5154b5 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.483084 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe80180-052d-43c6-88e6-b8811e5154b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:42.483399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe80180-052d-43c6-88e6-b8811e5154b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:42.484775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe80180-052d-43c6-88e6-b8811e5154b5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:42.491423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 447/902] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:42.492267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe80180-052d-43c6-88e6-b8811e5154b5 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:42.529999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:42.551893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3033df-f909-457f-909f-41fb938366b1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:42.594138 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92e0c9-f8b8-4735-bed3-12d8a9d1d5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:42.595229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92e0c9-f8b8-4735-bed3-12d8a9d1d5ad None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.595784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92e0c9-f8b8-4735-bed3-12d8a9d1d5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.596466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92e0c9-f8b8-4735-bed3-12d8a9d1d5ad None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.597941 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92e0c9-f8b8-4735-bed3-12d8a9d1d5ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:42.599869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92e0c9-f8b8-4735-bed3-12d8a9d1d5ad None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:42.608000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe80180-052d-43c6-88e6-b8811e5154b5 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:42.621295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 456/903] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:50:42 2022] POST /identity/v3/services => generated 241 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:42.622421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92e0c9-f8b8-4735-bed3-12d8a9d1d5ad None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:42.663960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1958e7-1d47-4c8a-97dc-074ada6ee386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:42.664701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1958e7-1d47-4c8a-97dc-074ada6ee386 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.664906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1958e7-1d47-4c8a-97dc-074ada6ee386 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.665190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1958e7-1d47-4c8a-97dc-074ada6ee386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.681709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:42.746569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1958e7-1d47-4c8a-97dc-074ada6ee386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:42.747992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1958e7-1d47-4c8a-97dc-074ada6ee386 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:42.757879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92e0c9-f8b8-4735-bed3-12d8a9d1d5ad None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:42.786519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:42.797570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed85207-2dd2-4e5d-80c0-4a844d0a5f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:42.798608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed85207-2dd2-4e5d-80c0-4a844d0a5f0b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.799056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed85207-2dd2-4e5d-80c0-4a844d0a5f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.799338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed85207-2dd2-4e5d-80c0-4a844d0a5f0b None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.800058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed85207-2dd2-4e5d-80c0-4a844d0a5f0b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:42.800396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed85207-2dd2-4e5d-80c0-4a844d0a5f0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:42.801806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed85207-2dd2-4e5d-80c0-4a844d0a5f0b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:42.807494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 457/904] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:42.814058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed85207-2dd2-4e5d-80c0-4a844d0a5f0b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:42.870571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 448/905] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:42 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:50:42.876659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1958e7-1d47-4c8a-97dc-074ada6ee386 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:42.936357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed85207-2dd2-4e5d-80c0-4a844d0a5f0b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:42.939575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:42.980966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab59eef-7f63-49a5-ac22-b9e7c4ac2e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:42.981691 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab59eef-7f63-49a5-ac22-b9e7c4ac2e8d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:42.981981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab59eef-7f63-49a5-ac22-b9e7c4ac2e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:42.982267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab59eef-7f63-49a5-ac22-b9e7c4ac2e8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:42.983636 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab59eef-7f63-49a5-ac22-b9e7c4ac2e8d None admin] RBAC: Adding query filter params (interface=public, service_id=ec8510529ad34bdba1566f6bd10dec68, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:42.983636 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab59eef-7f63-49a5-ac22-b9e7c4ac2e8d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:42.984852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab59eef-7f63-49a5-ac22-b9e7c4ac2e8d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:42.991688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 458/906] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jun 21 18:50:42 2022] GET /identity/v3/endpoints?service_id=ec8510529ad34bdba1566f6bd10dec68&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:42.996778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab59eef-7f63-49a5-ac22-b9e7c4ac2e8d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:43.025634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1958e7-1d47-4c8a-97dc-074ada6ee386 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:43.067321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:43.076838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62075d8a-621a-4296-af31-04840ad632e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:43.078344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62075d8a-621a-4296-af31-04840ad632e3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:43.078776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62075d8a-621a-4296-af31-04840ad632e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:43.079198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62075d8a-621a-4296-af31-04840ad632e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:43.098865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62075d8a-621a-4296-af31-04840ad632e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:43.100568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62075d8a-621a-4296-af31-04840ad632e3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:43.114916 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab59eef-7f63-49a5-ac22-b9e7c4ac2e8d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-62075d8a-621a-4296-af31-04840ad632e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:43.128822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:43.129981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:43.131092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:43.131092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:43.131092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:43.131092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:43.138475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 449/907] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:43.139353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62075d8a-621a-4296-af31-04840ad632e3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:43.160630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3471a-45c6-4315-8530-1501d4737421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:43.161342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3471a-45c6-4315-8530-1501d4737421 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:43.161632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3471a-45c6-4315-8530-1501d4737421 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:43.161915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3471a-45c6-4315-8530-1501d4737421 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:43.162609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3471a-45c6-4315-8530-1501d4737421 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:43.163983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3471a-45c6-4315-8530-1501d4737421 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:43.177232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 459/908] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jun 21 18:50:42 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:50:43.257790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62075d8a-621a-4296-af31-04840ad632e3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:43.297019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f502f3-d92e-41b9-949b-01df0a4ad5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:43.298161 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f502f3-d92e-41b9-949b-01df0a4ad5dd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:43.298658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f502f3-d92e-41b9-949b-01df0a4ad5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:43.299144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f502f3-d92e-41b9-949b-01df0a4ad5dd None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:43.300140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f502f3-d92e-41b9-949b-01df0a4ad5dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:43.300680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f502f3-d92e-41b9-949b-01df0a4ad5dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:43.302368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f502f3-d92e-41b9-949b-01df0a4ad5dd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:43.318218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 450/909] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:43.329831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3471a-45c6-4315-8530-1501d4737421 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:43.365783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:43.443908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3471a-45c6-4315-8530-1501d4737421 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:43.478478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55107f05-095f-40a0-b775-5e952ed8000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:43.479272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55107f05-095f-40a0-b775-5e952ed8000d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:43.479549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55107f05-095f-40a0-b775-5e952ed8000d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:43.479821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55107f05-095f-40a0-b775-5e952ed8000d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:43.486798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55107f05-095f-40a0-b775-5e952ed8000d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:43.488239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55107f05-095f-40a0-b775-5e952ed8000d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-55107f05-095f-40a0-b775-5e952ed8000d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:43.503207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:43.504162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:43.505180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:43.505180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:43.505180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:43.505180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:43.505180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:43.505180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:43.505180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:43.505180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:43.505180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 460/910] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:43.510405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f502f3-d92e-41b9-949b-01df0a4ad5dd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:43.558075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:43.622923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f502f3-d92e-41b9-949b-01df0a4ad5dd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:43.657728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2000244-c5b9-4286-a3ed-9483aa2dcb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:43.658539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2000244-c5b9-4286-a3ed-9483aa2dcb2c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:43.658916 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2000244-c5b9-4286-a3ed-9483aa2dcb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:43.659384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2000244-c5b9-4286-a3ed-9483aa2dcb2c None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:43.660818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2000244-c5b9-4286-a3ed-9483aa2dcb2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:43.661387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2000244-c5b9-4286-a3ed-9483aa2dcb2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:43.664039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2000244-c5b9-4286-a3ed-9483aa2dcb2c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:43.688304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 451/911] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:43.697937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55107f05-095f-40a0-b775-5e952ed8000d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:43.828070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:43.842732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55107f05-095f-40a0-b775-5e952ed8000d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:43.880785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911f766-abd5-4ea7-bf9c-0a6b00c504aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:43.881424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f766-abd5-4ea7-bf9c-0a6b00c504aa None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:43.881710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f766-abd5-4ea7-bf9c-0a6b00c504aa None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:43.881991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911f766-abd5-4ea7-bf9c-0a6b00c504aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:43.882859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911f766-abd5-4ea7-bf9c-0a6b00c504aa None admin] RBAC: Adding query filter params (role.id=c87ba8b8c46b4328bde6b43b256e79e3, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=9fee9b6e04f24f96adb1a70306463a91) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:43.883182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911f766-abd5-4ea7-bf9c-0a6b00c504aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:43.884699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911f766-abd5-4ea7-bf9c-0a6b00c504aa None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:43.891639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 461/912] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:43 2022] GET /identity/v3/role_assignments?user.id=9fee9b6e04f24f96adb1a70306463a91&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=c87ba8b8c46b4328bde6b43b256e79e3 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:43.933268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada61bdc-f258-45d6-b450-d6de840179db None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:43.933612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada61bdc-f258-45d6-b450-d6de840179db None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:43.933848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada61bdc-f258-45d6-b450-d6de840179db None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:43.935266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 452/913] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:43.941549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2934007-dd14-44cc-adb0-09794ddf4467 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:43.941900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2934007-dd14-44cc-adb0-09794ddf4467 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:43.942121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2934007-dd14-44cc-adb0-09794ddf4467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:43.960564 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c2934007-dd14-44cc-adb0-09794ddf4467 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:43.961840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c2934007-dd14-44cc-adb0-09794ddf4467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:44.049534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 462/914] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:44.065209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada61bdc-f258-45d6-b450-d6de840179db None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:44.181185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada61bdc-f258-45d6-b450-d6de840179db None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:44.182818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:44.225245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea6a39-f572-4398-bf2d-a90dba11bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:44.226066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea6a39-f572-4398-bf2d-a90dba11bf30 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:44.226355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea6a39-f572-4398-bf2d-a90dba11bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:44.226643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea6a39-f572-4398-bf2d-a90dba11bf30 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:44.227311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea6a39-f572-4398-bf2d-a90dba11bf30 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:44.228747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea6a39-f572-4398-bf2d-a90dba11bf30 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-eaea6a39-f572-4398-bf2d-a90dba11bf30 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:44.244588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:44.245798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:44.246750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 453/915] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:50:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:44.252060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2934007-dd14-44cc-adb0-09794ddf4467 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:44.344747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:44.361829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2934007-dd14-44cc-adb0-09794ddf4467 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:44.401150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84a718-0b13-4900-ba00-e6b046643596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:44.401834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84a718-0b13-4900-ba00-e6b046643596 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:44.402123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84a718-0b13-4900-ba00-e6b046643596 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:44.402406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84a718-0b13-4900-ba00-e6b046643596 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:44.403131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84a718-0b13-4900-ba00-e6b046643596 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:44.403444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84a718-0b13-4900-ba00-e6b046643596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:44.404840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84a718-0b13-4900-ba00-e6b046643596 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:44.414353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 463/916] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:50:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:44.418240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea6a39-f572-4398-bf2d-a90dba11bf30 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:44.531622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea6a39-f572-4398-bf2d-a90dba11bf30 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:44.566710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073784d-fe1c-47d2-84df-e0ebc4d6a8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:44.567380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073784d-fe1c-47d2-84df-e0ebc4d6a8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:44.567664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073784d-fe1c-47d2-84df-e0ebc4d6a8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:44.567941 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073784d-fe1c-47d2-84df-e0ebc4d6a8d0 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:44.568648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073784d-fe1c-47d2-84df-e0ebc4d6a8d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:44.568958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073784d-fe1c-47d2-84df-e0ebc4d6a8d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:44.570359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073784d-fe1c-47d2-84df-e0ebc4d6a8d0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:44.576348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 454/917] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jun 21 18:50:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:44.581771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84a718-0b13-4900-ba00-e6b046643596 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:44.688065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84a718-0b13-4900-ba00-e6b046643596 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:44.726774 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88a9c9-60c4-469c-a8be-8a37da49a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:44.727576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88a9c9-60c4-469c-a8be-8a37da49a785 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:44.727949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88a9c9-60c4-469c-a8be-8a37da49a785 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:44.728233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88a9c9-60c4-469c-a8be-8a37da49a785 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:44.729059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88a9c9-60c4-469c-a8be-8a37da49a785 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:44.730541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88a9c9-60c4-469c-a8be-8a37da49a785 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:44.757269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 464/918] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jun 21 18:50:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:44.888629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b6d6e-a84e-4978-b075-a5cda877c739 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:44.889318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b6d6e-a84e-4978-b075-a5cda877c739 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:44.889318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b6d6e-a84e-4978-b075-a5cda877c739 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:44.890886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 455/919] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:44.896820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b350c-0421-4407-917d-0ff531f38b97 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:44.897155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b350c-0421-4407-917d-0ff531f38b97 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:44.897415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b350c-0421-4407-917d-0ff531f38b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:44.903766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:44.918972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b55b350c-0421-4407-917d-0ff531f38b97 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:44.920180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b55b350c-0421-4407-917d-0ff531f38b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:44.999032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 465/920] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:45.024248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b6d6e-a84e-4978-b075-a5cda877c739 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:45.060968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:45.142452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b6d6e-a84e-4978-b075-a5cda877c739 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:45.179886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2bc09-c8e7-476b-a7da-33e5610e548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:45.180691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2bc09-c8e7-476b-a7da-33e5610e548a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:45.181572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2bc09-c8e7-476b-a7da-33e5610e548a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:45.181851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2bc09-c8e7-476b-a7da-33e5610e548a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:45.182528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2bc09-c8e7-476b-a7da-33e5610e548a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:45.183919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2bc09-c8e7-476b-a7da-33e5610e548a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-79c2bc09-c8e7-476b-a7da-33e5610e548a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:45.196734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:45.199087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:45.200139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 456/921] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:45.206372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b350c-0421-4407-917d-0ff531f38b97 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:45.317389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b350c-0421-4407-917d-0ff531f38b97 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:45.349505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:45.355279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d6e2c-889a-4771-bf9d-80cdd897f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:45.356180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d6e2c-889a-4771-bf9d-80cdd897f57b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:45.356657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d6e2c-889a-4771-bf9d-80cdd897f57b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:45.357003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d6e2c-889a-4771-bf9d-80cdd897f57b None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:45.357944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d6e2c-889a-4771-bf9d-80cdd897f57b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:45.358257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d6e2c-889a-4771-bf9d-80cdd897f57b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:45.359651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d6e2c-889a-4771-bf9d-80cdd897f57b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:45.365359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 466/922] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:45.371354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2bc09-c8e7-476b-a7da-33e5610e548a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:45.377701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aadb4-9ebe-4f9b-977d-17a5a3e853bc None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:45.378304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aadb4-9ebe-4f9b-977d-17a5a3e853bc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:45.378906 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aadb4-9ebe-4f9b-977d-17a5a3e853bc None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:45.380796 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 467/923] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:45.387242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352948-307a-4b0f-934a-7c5ce9ec87ea None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:45.387625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352948-307a-4b0f-934a-7c5ce9ec87ea None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:45.387907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352948-307a-4b0f-934a-7c5ce9ec87ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:45.410005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f7352948-307a-4b0f-934a-7c5ce9ec87ea None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:45.411445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f7352948-307a-4b0f-934a-7c5ce9ec87ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:45.489461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:45.498920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2bc09-c8e7-476b-a7da-33e5610e548a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:45.525446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 468/924] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:45.546161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352948-307a-4b0f-934a-7c5ce9ec87ea None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:45.550083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad852c0-787e-4f1a-94f9-0982ab552159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:45.550083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad852c0-787e-4f1a-94f9-0982ab552159 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:45.550461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad852c0-787e-4f1a-94f9-0982ab552159 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:45.550660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad852c0-787e-4f1a-94f9-0982ab552159 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:45.553727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad852c0-787e-4f1a-94f9-0982ab552159 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:45.554293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad852c0-787e-4f1a-94f9-0982ab552159 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:45.557249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad852c0-787e-4f1a-94f9-0982ab552159 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:45.565737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 457/925] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:45.573392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad852c0-787e-4f1a-94f9-0982ab552159 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:45.676351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352948-307a-4b0f-934a-7c5ce9ec87ea None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:45.705431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:45.714446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad852c0-787e-4f1a-94f9-0982ab552159 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:45.719664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797f702-a2b0-4cf1-9bee-c0a367ef068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:45.720616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797f702-a2b0-4cf1-9bee-c0a367ef068c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:45.720997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797f702-a2b0-4cf1-9bee-c0a367ef068c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:45.721400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797f702-a2b0-4cf1-9bee-c0a367ef068c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:45.722243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797f702-a2b0-4cf1-9bee-c0a367ef068c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:45.723792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797f702-a2b0-4cf1-9bee-c0a367ef068c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a797f702-a2b0-4cf1-9bee-c0a367ef068c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:45.738154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:45.739952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 18:50:45.740988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:45.748264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 469/926] 10.222.0.70 () {56 vars in 1132 bytes} [Tue Jun 21 18:50:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:45.749766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797f702-a2b0-4cf1-9bee-c0a367ef068c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:45.775017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d497f8-b568-4ba2-b35a-57924e0c7152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:45.776365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d497f8-b568-4ba2-b35a-57924e0c7152 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:45.777200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d497f8-b568-4ba2-b35a-57924e0c7152 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:45.777869 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d497f8-b568-4ba2-b35a-57924e0c7152 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:45.779517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d497f8-b568-4ba2-b35a-57924e0c7152 None admin] RBAC: Adding query filter params (interface=public, service_id=4afaacad8d4b4696b79caec3a97b8a17, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:45.780087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d497f8-b568-4ba2-b35a-57924e0c7152 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:45.783061 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d497f8-b568-4ba2-b35a-57924e0c7152 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:45.797049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 458/927] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jun 21 18:50:45 2022] GET /identity/v3/endpoints?service_id=4afaacad8d4b4696b79caec3a97b8a17&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:45.799223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d497f8-b568-4ba2-b35a-57924e0c7152 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:45.841916 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:45.906705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797f702-a2b0-4cf1-9bee-c0a367ef068c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:45.948224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd2b2f-99ec-481a-8388-340dd649e1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:45.949353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2b2f-99ec-481a-8388-340dd649e1ba None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:45.949865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2b2f-99ec-481a-8388-340dd649e1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:45.950351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2b2f-99ec-481a-8388-340dd649e1ba None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:45.951269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd2b2f-99ec-481a-8388-340dd649e1ba None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:45.951778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd2b2f-99ec-481a-8388-340dd649e1ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:45.953375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd2b2f-99ec-481a-8388-340dd649e1ba None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:45.957122 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d497f8-b568-4ba2-b35a-57924e0c7152 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:45.959866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 470/928] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:45.969125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd2b2f-99ec-481a-8388-340dd649e1ba None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:46.009147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81dd87f-25a2-4017-992b-189da00dcbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:46.009909 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dd87f-25a2-4017-992b-189da00dcbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.010245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dd87f-25a2-4017-992b-189da00dcbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.010779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dd87f-25a2-4017-992b-189da00dcbe0 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.011525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81dd87f-25a2-4017-992b-189da00dcbe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:46.012955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81dd87f-25a2-4017-992b-189da00dcbe0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:46.020165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 459/929] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jun 21 18:50:45 2022] GET /identity/v3/services => generated 1497 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:50:46.034778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:46.044084 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dee60-575e-42c5-827d-94bac193ebbe None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.044449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dee60-575e-42c5-827d-94bac193ebbe None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.044665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dee60-575e-42c5-827d-94bac193ebbe None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.049543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 460/930] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:46.053375 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0b8a6-785a-4f02-beca-adf9be86f213 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.053756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0b8a6-785a-4f02-beca-adf9be86f213 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.054243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0b8a6-785a-4f02-beca-adf9be86f213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.076899 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-baa0b8a6-785a-4f02-beca-adf9be86f213 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:46.078444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-baa0b8a6-785a-4f02-beca-adf9be86f213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:46.109966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd2b2f-99ec-481a-8388-340dd649e1ba None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:46.168855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d68eb-d87c-4202-abb6-47928e892aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:46.169617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d68eb-d87c-4202-abb6-47928e892aee None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.170478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d68eb-d87c-4202-abb6-47928e892aee None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.170774 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d68eb-d87c-4202-abb6-47928e892aee None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.171523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d68eb-d87c-4202-abb6-47928e892aee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:46.171841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d68eb-d87c-4202-abb6-47928e892aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:46.173732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d68eb-d87c-4202-abb6-47928e892aee None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:46.184791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 471/931] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:46.185467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 461/932] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:46.185778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d68eb-d87c-4202-abb6-47928e892aee None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:46.202290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0b8a6-785a-4f02-beca-adf9be86f213 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:46.222600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:46.324987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d68eb-d87c-4202-abb6-47928e892aee None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:46.335445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0b8a6-785a-4f02-beca-adf9be86f213 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:46.344356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:46.371423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b1a82-e627-4d19-9a96-016775036a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:46.372092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b1a82-e627-4d19-9a96-016775036a51 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.372376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b1a82-e627-4d19-9a96-016775036a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.372653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b1a82-e627-4d19-9a96-016775036a51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.373471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b1a82-e627-4d19-9a96-016775036a51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:46.374913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b1a82-e627-4d19-9a96-016775036a51 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:46.378854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6a4fe-77cc-4332-877f-f81f182f505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:46.379873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6a4fe-77cc-4332-877f-f81f182f505f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.380159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6a4fe-77cc-4332-877f-f81f182f505f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.380791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6a4fe-77cc-4332-877f-f81f182f505f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.394807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6a4fe-77cc-4332-877f-f81f182f505f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:46.401544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 472/933] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jun 21 18:50:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-84b6a4fe-77cc-4332-877f-f81f182f505f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:46.403398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:46.404429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:50:46.405352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:46.413209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 462/934] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jun 21 18:50:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:46.413444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b1a82-e627-4d19-9a96-016775036a51 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:46.558383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b1a82-e627-4d19-9a96-016775036a51 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:46.598290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ff8b8-eb88-4ae1-ba1d-19430b37dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:46.599290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ff8b8-eb88-4ae1-ba1d-19430b37dba2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.599570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ff8b8-eb88-4ae1-ba1d-19430b37dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.599840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ff8b8-eb88-4ae1-ba1d-19430b37dba2 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.600591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ff8b8-eb88-4ae1-ba1d-19430b37dba2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:46.600896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ff8b8-eb88-4ae1-ba1d-19430b37dba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:46.602309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ff8b8-eb88-4ae1-ba1d-19430b37dba2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:46.616572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 473/935] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jun 21 18:50:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:46.625928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6a4fe-77cc-4332-877f-f81f182f505f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:46.723385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:46.751052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6a4fe-77cc-4332-877f-f81f182f505f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:46.792700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cba949-c77f-42fa-9ff7-a9315588ee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:46.793579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cba949-c77f-42fa-9ff7-a9315588ee9b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.793862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cba949-c77f-42fa-9ff7-a9315588ee9b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.794137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cba949-c77f-42fa-9ff7-a9315588ee9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.805654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cba949-c77f-42fa-9ff7-a9315588ee9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:46.807914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cba949-c77f-42fa-9ff7-a9315588ee9b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a6cba949-c77f-42fa-9ff7-a9315588ee9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:46.826372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:46.827511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:46.828494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:46.828494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:46.828494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:46.828494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:46.828494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 463/936] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:46.833264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ff8b8-eb88-4ae1-ba1d-19430b37dba2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:46.870543 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:46.948778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ff8b8-eb88-4ae1-ba1d-19430b37dba2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:46.984514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d4798-1e89-4acc-9079-aec7ce921fe8 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.985031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d4798-1e89-4acc-9079-aec7ce921fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.985156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d4798-1e89-4acc-9079-aec7ce921fe8 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.986095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf50509-8e47-41b1-a3df-5be30ccde764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:46.986963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf50509-8e47-41b1-a3df-5be30ccde764 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.987442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf50509-8e47-41b1-a3df-5be30ccde764 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.987916 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf50509-8e47-41b1-a3df-5be30ccde764 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:46.988829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf50509-8e47-41b1-a3df-5be30ccde764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:46.989351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf50509-8e47-41b1-a3df-5be30ccde764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:46.990455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 464/937] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:46.991155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf50509-8e47-41b1-a3df-5be30ccde764 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:46.991984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096e9b1-12a8-402b-9d8e-2107fbc1fb0c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:46.992281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096e9b1-12a8-402b-9d8e-2107fbc1fb0c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:46.992564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096e9b1-12a8-402b-9d8e-2107fbc1fb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:47.005930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 474/938] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:47.011471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf50509-8e47-41b1-a3df-5be30ccde764 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:47.012805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0096e9b1-12a8-402b-9d8e-2107fbc1fb0c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:47.014059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0096e9b1-12a8-402b-9d8e-2107fbc1fb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:47.105676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 465/939] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:47.121317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:47.129293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096e9b1-12a8-402b-9d8e-2107fbc1fb0c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:47.133240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf50509-8e47-41b1-a3df-5be30ccde764 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:47.172969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76ae6-b921-4057-9c7f-2bbfb1ea863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:47.174079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76ae6-b921-4057-9c7f-2bbfb1ea863c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:47.174592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76ae6-b921-4057-9c7f-2bbfb1ea863c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:47.175021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76ae6-b921-4057-9c7f-2bbfb1ea863c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:47.183654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b76ae6-b921-4057-9c7f-2bbfb1ea863c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:47.185441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b76ae6-b921-4057-9c7f-2bbfb1ea863c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:47.193172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-69b76ae6-b921-4057-9c7f-2bbfb1ea863c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:47.205343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:47.206570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:47.207749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:47.215399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 475/940] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:50:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:47.216712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76ae6-b921-4057-9c7f-2bbfb1ea863c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:47.242932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096e9b1-12a8-402b-9d8e-2107fbc1fb0c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:47.281681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:47.282675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:47.283228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:47.283833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:47.284673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:47.286495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:47.288547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WARNING keystone.server.flask.common [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 21 18:50:47.310939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.sql.core [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 18:50:47.310939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 18:50:47.310939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [parameters: {'id': '0463b5ff13bb446b9f1158dccdcd72c4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 18:50:47.310939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80157) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0463b5ff13bb446b9f1158dccdcd72c4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0463b5ff13bb446b9f1158dccdcd72c4. Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:50:47.331734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result.read() Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 21 18:50:47.332746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application yield resource Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise self.value Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application session.commit() Jun 21 18:50:47.333799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.session.flush() Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application compat.raise_( Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 18:50:47.334688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 18:50:47.335609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result.read() Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:50:47.336394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [parameters: {'id': '0463b5ff13bb446b9f1158dccdcd72c4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:47.337904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:47.338690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0463b5ff13bb446b9f1158dccdcd72c4. Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:47.339594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 466/941] 10.222.0.70 () {60 vars in 1140 bytes} [Tue Jun 21 18:50:47 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:47.342623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:47.343629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76ae6-b921-4057-9c7f-2bbfb1ea863c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:47.382410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f9378-48bf-494f-9803-d6b34c115b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:47.383463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f9378-48bf-494f-9803-d6b34c115b11 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:47.384563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f9378-48bf-494f-9803-d6b34c115b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:47.385080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f9378-48bf-494f-9803-d6b34c115b11 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:47.386114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f9378-48bf-494f-9803-d6b34c115b11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:47.386615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f9378-48bf-494f-9803-d6b34c115b11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:47.388253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f9378-48bf-494f-9803-d6b34c115b11 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:47.413216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 476/942] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jun 21 18:50:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:47.414463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f9378-48bf-494f-9803-d6b34c115b11 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:47.427006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:47.463016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607817-42b6-4dfd-9152-c3a96034b1f3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:47.478893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:47.503808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a73dcf-efad-438f-8324-8f4dcb3d7f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:47.504899 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a73dcf-efad-438f-8324-8f4dcb3d7f6e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:47.505392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a73dcf-efad-438f-8324-8f4dcb3d7f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:47.505915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a73dcf-efad-438f-8324-8f4dcb3d7f6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:47.514805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a73dcf-efad-438f-8324-8f4dcb3d7f6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:47.516759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a73dcf-efad-438f-8324-8f4dcb3d7f6e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:47.533461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f9378-48bf-494f-9803-d6b34c115b11 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c4a73dcf-efad-438f-8324-8f4dcb3d7f6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:47.534310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:47.535358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:47.536465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:47.536465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:47.536465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:47.536465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:47.536465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:47.536465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:47.536465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:47.536465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:47.538906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 467/943] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:47.549110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a73dcf-efad-438f-8324-8f4dcb3d7f6e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:47.580863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c683a-e747-4edc-b3c8-f7e1c381c810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:47.581948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c683a-e747-4edc-b3c8-f7e1c381c810 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:47.582233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c683a-e747-4edc-b3c8-f7e1c381c810 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:47.582517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c683a-e747-4edc-b3c8-f7e1c381c810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:47.593365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c683a-e747-4edc-b3c8-f7e1c381c810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:47.594822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c683a-e747-4edc-b3c8-f7e1c381c810 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-1d2c683a-e747-4edc-b3c8-f7e1c381c810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:47.624008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:47.625251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:47.626278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:47.626278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:47.626278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:47.626278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:47.626278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 477/944] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:47.631241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c683a-e747-4edc-b3c8-f7e1c381c810 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:47.667027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a73dcf-efad-438f-8324-8f4dcb3d7f6e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:47.695821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:47.720362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc891c-91f7-4f58-993c-d1fed8e41e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:47.722002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc891c-91f7-4f58-993c-d1fed8e41e7e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:47.722529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc891c-91f7-4f58-993c-d1fed8e41e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:47.723544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc891c-91f7-4f58-993c-d1fed8e41e7e None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:47.725828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc891c-91f7-4f58-993c-d1fed8e41e7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:47.726544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc891c-91f7-4f58-993c-d1fed8e41e7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:47.729026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc891c-91f7-4f58-993c-d1fed8e41e7e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:47.750020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c683a-e747-4edc-b3c8-f7e1c381c810 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:47.761978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 468/945] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:47.802382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda305f-68b0-46e1-9d1b-15c536ad6ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:47.803445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda305f-68b0-46e1-9d1b-15c536ad6ebc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:47.803909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda305f-68b0-46e1-9d1b-15c536ad6ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:47.804220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda305f-68b0-46e1-9d1b-15c536ad6ebc None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:47.804971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda305f-68b0-46e1-9d1b-15c536ad6ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:47.805317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda305f-68b0-46e1-9d1b-15c536ad6ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:47.807476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda305f-68b0-46e1-9d1b-15c536ad6ebc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:47.825060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 478/946] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:47.830826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc891c-91f7-4f58-993c-d1fed8e41e7e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:47.873287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:47.950588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc891c-91f7-4f58-993c-d1fed8e41e7e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:47.990409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76648c-b3d5-4d67-ad70-6f9bad8c0c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:47.991237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76648c-b3d5-4d67-ad70-6f9bad8c0c35 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:47.991518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76648c-b3d5-4d67-ad70-6f9bad8c0c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:47.991799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76648c-b3d5-4d67-ad70-6f9bad8c0c35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.000219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76648c-b3d5-4d67-ad70-6f9bad8c0c35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:48.001796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76648c-b3d5-4d67-ad70-6f9bad8c0c35 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-1f76648c-b3d5-4d67-ad70-6f9bad8c0c35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:48.017020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:48.018251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:48.019836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:48.019836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:48.019836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:48.019836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:48.019836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:48.019836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:48.019836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:48.019836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:48.019836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 469/947] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:48.024653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda305f-68b0-46e1-9d1b-15c536ad6ebc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:48.136164 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda305f-68b0-46e1-9d1b-15c536ad6ebc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:48.157785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:48.173427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bbeb5-8f71-4375-861a-666e975e3dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:48.174097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bbeb5-8f71-4375-861a-666e975e3dad None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:48.174383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bbeb5-8f71-4375-861a-666e975e3dad None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:48.174663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bbeb5-8f71-4375-861a-666e975e3dad None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.175468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bbeb5-8f71-4375-861a-666e975e3dad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:48.175779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bbeb5-8f71-4375-861a-666e975e3dad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:48.177249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bbeb5-8f71-4375-861a-666e975e3dad None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:48.191753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 479/948] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:48.197720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76648c-b3d5-4d67-ad70-6f9bad8c0c35 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:48.291327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce0867-8e27-4897-8914-f85e6068e1e9 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:48.291529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce0867-8e27-4897-8914-f85e6068e1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:48.291803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce0867-8e27-4897-8914-f85e6068e1e9 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.293520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 480/949] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:48.302122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5333f-7281-4ac2-8273-758cc1d3eeac None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:48.302620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:48.303013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5333f-7281-4ac2-8273-758cc1d3eeac None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:48.303502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5333f-7281-4ac2-8273-758cc1d3eeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.312937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76648c-b3d5-4d67-ad70-6f9bad8c0c35 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:48.322559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0cb5333f-7281-4ac2-8273-758cc1d3eeac None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:48.324255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0cb5333f-7281-4ac2-8273-758cc1d3eeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:48.358115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4c280-bc44-4299-8f61-84b1f383cb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:48.359030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4c280-bc44-4299-8f61-84b1f383cb82 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:48.359321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4c280-bc44-4299-8f61-84b1f383cb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:48.359777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4c280-bc44-4299-8f61-84b1f383cb82 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/9fee9b6e04f24f96adb1a70306463a91/roles/c87ba8b8c46b4328bde6b43b256e79e3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.363304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4c280-bc44-4299-8f61-84b1f383cb82 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=9fee9b6e04f24f96adb1a70306463a91, role_id=c87ba8b8c46b4328bde6b43b256e79e3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:48.365005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4c280-bc44-4299-8f61-84b1f383cb82 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:48.388797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 470/950] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:50:48 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/9fee9b6e04f24f96adb1a70306463a91/roles/c87ba8b8c46b4328bde6b43b256e79e3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:50:48.425431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 481/951] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:48.444033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4c280-bc44-4299-8f61-84b1f383cb82 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:48.573887 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4c280-bc44-4299-8f61-84b1f383cb82 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:48.615072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77263176-ec24-4e77-b13f-3ab8afc6d194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:48.615954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77263176-ec24-4e77-b13f-3ab8afc6d194 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:48.616244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77263176-ec24-4e77-b13f-3ab8afc6d194 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:48.616525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77263176-ec24-4e77-b13f-3ab8afc6d194 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.617245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77263176-ec24-4e77-b13f-3ab8afc6d194 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:48.618726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77263176-ec24-4e77-b13f-3ab8afc6d194 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:48.624535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd62fdc-d08d-42a5-8cbd-b1b6fef6b7fd None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:48.624828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd62fdc-d08d-42a5-8cbd-b1b6fef6b7fd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:48.625135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd62fdc-d08d-42a5-8cbd-b1b6fef6b7fd None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.627050 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 482/952] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:48.634935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7962692-58c4-422a-8206-db0f6c7bbbc0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:48.635229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7962692-58c4-422a-8206-db0f6c7bbbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:48.635507 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7962692-58c4-422a-8206-db0f6c7bbbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-77263176-ec24-4e77-b13f-3ab8afc6d194 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:48.635890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:48.636992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:48.637976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:48.640475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 471/953] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:48.646561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77263176-ec24-4e77-b13f-3ab8afc6d194 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:48.658692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a7962692-58c4-422a-8206-db0f6c7bbbc0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:48.660261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a7962692-58c4-422a-8206-db0f6c7bbbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:48.670085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:48.744196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 483/954] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:48.767074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7962692-58c4-422a-8206-db0f6c7bbbc0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:48.788498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77263176-ec24-4e77-b13f-3ab8afc6d194 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:48.820142 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:48.838460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece1c13-922d-4232-9cb9-f8037c6700c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:48.839434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece1c13-922d-4232-9cb9-f8037c6700c1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:48.839838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece1c13-922d-4232-9cb9-f8037c6700c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:48.840193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece1c13-922d-4232-9cb9-f8037c6700c1 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.841237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece1c13-922d-4232-9cb9-f8037c6700c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:48.841903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece1c13-922d-4232-9cb9-f8037c6700c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:48.843619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece1c13-922d-4232-9cb9-f8037c6700c1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:48.850989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 472/955] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:48.857708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece1c13-922d-4232-9cb9-f8037c6700c1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:48.909828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7962692-58c4-422a-8206-db0f6c7bbbc0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:48.960796 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0682638-1521-4072-97a0-7559f4cd856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:48.961767 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0682638-1521-4072-97a0-7559f4cd856c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:48.962850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0682638-1521-4072-97a0-7559f4cd856c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:48.963134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0682638-1521-4072-97a0-7559f4cd856c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:48.963810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0682638-1521-4072-97a0-7559f4cd856c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:48.966063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0682638-1521-4072-97a0-7559f4cd856c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b0682638-1521-4072-97a0-7559f4cd856c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:48.980935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:48.982064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:48.982933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 484/956] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:48.992888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0682638-1521-4072-97a0-7559f4cd856c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:48.999640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:49.023964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece1c13-922d-4232-9cb9-f8037c6700c1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:49.062731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf1fbb-f972-4a63-9ae0-49b9e5f7b7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:49.063993 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf1fbb-f972-4a63-9ae0-49b9e5f7b7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:49.064647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf1fbb-f972-4a63-9ae0-49b9e5f7b7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:49.065190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf1fbb-f972-4a63-9ae0-49b9e5f7b7b6 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:49.066064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf1fbb-f972-4a63-9ae0-49b9e5f7b7b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:49.066564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf1fbb-f972-4a63-9ae0-49b9e5f7b7b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:49.068717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf1fbb-f972-4a63-9ae0-49b9e5f7b7b6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:49.075556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 473/957] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:49.081616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf1fbb-f972-4a63-9ae0-49b9e5f7b7b6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:49.116063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0682638-1521-4072-97a0-7559f4cd856c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:49.125458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:49.162282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf815b5-c106-447e-afc0-cd179f78de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:49.162955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf815b5-c106-447e-afc0-cd179f78de5c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:49.163243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf815b5-c106-447e-afc0-cd179f78de5c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:49.163531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf815b5-c106-447e-afc0-cd179f78de5c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:49.164241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf815b5-c106-447e-afc0-cd179f78de5c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:49.164600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf815b5-c106-447e-afc0-cd179f78de5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:49.166024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf815b5-c106-447e-afc0-cd179f78de5c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:49.172542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 485/958] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:49.177839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf815b5-c106-447e-afc0-cd179f78de5c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:49.228670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf1fbb-f972-4a63-9ae0-49b9e5f7b7b6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:49.275780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99086a6-4a52-434e-b826-d176e2c16b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:49.277107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99086a6-4a52-434e-b826-d176e2c16b5f None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:49.277593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99086a6-4a52-434e-b826-d176e2c16b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:49.278171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99086a6-4a52-434e-b826-d176e2c16b5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:49.279428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99086a6-4a52-434e-b826-d176e2c16b5f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:49.281288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99086a6-4a52-434e-b826-d176e2c16b5f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:49.294887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf815b5-c106-447e-afc0-cd179f78de5c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:49.317319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 474/959] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jun 21 18:50:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:50:49.342838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203d869-2c83-43e2-ac58-337b6c779422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:49.343916 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203d869-2c83-43e2-ac58-337b6c779422 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:49.344463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203d869-2c83-43e2-ac58-337b6c779422 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:49.344944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203d869-2c83-43e2-ac58-337b6c779422 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:49.345913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203d869-2c83-43e2-ac58-337b6c779422 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:49.346491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203d869-2c83-43e2-ac58-337b6c779422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:49.348120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203d869-2c83-43e2-ac58-337b6c779422 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:49.354326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 486/960] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:50.002313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4e6b1-a733-4c39-97cc-16e596f98b60 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:50.002844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4e6b1-a733-4c39-97cc-16e596f98b60 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:50.002924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4e6b1-a733-4c39-97cc-16e596f98b60 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:50.004404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 475/961] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:50.010085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6d50c-dbf0-49fc-bed2-720924ad3664 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:50.010701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6d50c-dbf0-49fc-bed2-720924ad3664 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:50.011203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6d50c-dbf0-49fc-bed2-720924ad3664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:50.030095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-53e6d50c-dbf0-49fc-bed2-720924ad3664 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:50.031655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-53e6d50c-dbf0-49fc-bed2-720924ad3664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:50.059943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:50.110327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 487/962] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:50.135140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4e6b1-a733-4c39-97cc-16e596f98b60 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:50.138245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:50.246663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4e6b1-a733-4c39-97cc-16e596f98b60 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:50.282940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:50.283909 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:50.284369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:50.284824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:50.285757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:50.287395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:50.301262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.sql.core [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 18:50:50.301262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 18:50:50.301262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [parameters: {'user_id': '01940009722847dda48b516f37b3121e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 18:50:50.301262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80157) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result.read() Jun 21 18:50:50.309975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:50:50.310923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application yield resource Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application next(self.gen) Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise self.value Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application session.commit() Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 18:50:50.311901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.session.flush() Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application compat.raise_( Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 18:50:50.313065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 18:50:50.314006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application conn.query(q) Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application result.read() Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 18:50:50.314937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [parameters: {'user_id': '01940009722847dda48b516f37b3121e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:50.315783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 21 18:50:50.316574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 21 18:50:50.317729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 21 18:50:50.317729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 21 18:50:50.317729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 18:50:50.317729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 18:50:50.317729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 18:50:50.317729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:50.319897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 476/963] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jun 21 18:50:50 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:50.334229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6d50c-dbf0-49fc-bed2-720924ad3664 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:50.443712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6d50c-dbf0-49fc-bed2-720924ad3664 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:50.481545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9d884-467c-4402-b338-06102471aa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:50.482353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9d884-467c-4402-b338-06102471aa00 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:50.482640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9d884-467c-4402-b338-06102471aa00 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:50.482923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9d884-467c-4402-b338-06102471aa00 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:50.490845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9d884-467c-4402-b338-06102471aa00 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:50.492307 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9d884-467c-4402-b338-06102471aa00 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-66a9d884-467c-4402-b338-06102471aa00 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:50.507960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:50.509566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:50.510963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 488/964] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:50:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:50.518286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:50.597896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:50.603875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd124c4f-b85b-4cc9-8213-f1d2d76f5e75 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:50.604228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd124c4f-b85b-4cc9-8213-f1d2d76f5e75 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:50.604442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd124c4f-b85b-4cc9-8213-f1d2d76f5e75 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:50.606159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 489/965] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:50.611927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c5e05-a4b2-4e7e-8361-7e03a54094bb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:50.612222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c5e05-a4b2-4e7e-8361-7e03a54094bb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:50.612493 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c5e05-a4b2-4e7e-8361-7e03a54094bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:50.629899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b31c5e05-a4b2-4e7e-8361-7e03a54094bb None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:50.631104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b31c5e05-a4b2-4e7e-8361-7e03a54094bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:50.637839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767343e-b483-4f85-b71d-2c5337ada124 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:50.646592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:50.675489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830d260-af71-40f6-9dbe-eb800e130e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:50.676170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830d260-af71-40f6-9dbe-eb800e130e1d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:50.676460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830d260-af71-40f6-9dbe-eb800e130e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:50.676743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830d260-af71-40f6-9dbe-eb800e130e1d None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:50.677597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830d260-af71-40f6-9dbe-eb800e130e1d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:50.677910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830d260-af71-40f6-9dbe-eb800e130e1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:50.679593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830d260-af71-40f6-9dbe-eb800e130e1d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:50.715883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 490/966] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:50.729277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 477/967] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jun 21 18:50:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:50.736086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c5e05-a4b2-4e7e-8361-7e03a54094bb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:50.851576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c5e05-a4b2-4e7e-8361-7e03a54094bb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:50.892412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0e3fb-ceaa-46d1-9fca-753e4c6b67b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:50.893439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0e3fb-ceaa-46d1-9fca-753e4c6b67b7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:50.893732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0e3fb-ceaa-46d1-9fca-753e4c6b67b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:50.894023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0e3fb-ceaa-46d1-9fca-753e4c6b67b7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:50.908782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0e3fb-ceaa-46d1-9fca-753e4c6b67b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-19e0e3fb-ceaa-46d1-9fca-753e4c6b67b7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:50:50.917873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:50.918895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:50.919792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 491/968] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jun 21 18:50:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:50.925711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830d260-af71-40f6-9dbe-eb800e130e1d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:51.036754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830d260-af71-40f6-9dbe-eb800e130e1d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:51.060440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:51.077753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadc26f-3600-4322-8606-1c4499f0f92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:51.078432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadc26f-3600-4322-8606-1c4499f0f92f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.078745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadc26f-3600-4322-8606-1c4499f0f92f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.079026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadc26f-3600-4322-8606-1c4499f0f92f None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.079789 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadc26f-3600-4322-8606-1c4499f0f92f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:51.080107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadc26f-3600-4322-8606-1c4499f0f92f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:51.085787 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadc26f-3600-4322-8606-1c4499f0f92f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:51.095038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 478/969] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jun 21 18:50:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:51.100480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0e3fb-ceaa-46d1-9fca-753e4c6b67b7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:51.112363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:51.211017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0e3fb-ceaa-46d1-9fca-753e4c6b67b7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:51.247000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a3465-ef1c-458d-abe2-efaabb2d0287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:51.247799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3465-ef1c-458d-abe2-efaabb2d0287 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.248081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3465-ef1c-458d-abe2-efaabb2d0287 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.248355 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3465-ef1c-458d-abe2-efaabb2d0287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.260975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a3465-ef1c-458d-abe2-efaabb2d0287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:51.262974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a3465-ef1c-458d-abe2-efaabb2d0287 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-883a3465-ef1c-458d-abe2-efaabb2d0287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:51.282083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:51.283298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:51.284397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:51.284397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:51.284397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:51.284397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:51.287968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 492/970] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:51.289222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadc26f-3600-4322-8606-1c4499f0f92f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:51.417079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadc26f-3600-4322-8606-1c4499f0f92f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:51.457807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f29873-974c-4038-8fab-4dcf0cfcbb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:51.458570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f29873-974c-4038-8fab-4dcf0cfcbb9a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.458859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f29873-974c-4038-8fab-4dcf0cfcbb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.459142 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f29873-974c-4038-8fab-4dcf0cfcbb9a None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.459882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f29873-974c-4038-8fab-4dcf0cfcbb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:51.460197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f29873-974c-4038-8fab-4dcf0cfcbb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:51.461646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f29873-974c-4038-8fab-4dcf0cfcbb9a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:51.480213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 479/971] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:51.481832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a3465-ef1c-458d-abe2-efaabb2d0287 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:51.492489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd99975-a216-4e37-a339-4a7f0d496bc3 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.492489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd99975-a216-4e37-a339-4a7f0d496bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.492489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd99975-a216-4e37-a339-4a7f0d496bc3 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.494114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 480/972] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:51.499043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30d41c-31c3-4513-9a4b-5448c63885ce None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.499342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30d41c-31c3-4513-9a4b-5448c63885ce None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.499630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30d41c-31c3-4513-9a4b-5448c63885ce None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.501127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 481/973] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:51.502103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077365-9604-4d4f-84c2-6ab8bb1124dd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.502482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077365-9604-4d4f-84c2-6ab8bb1124dd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.502774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077365-9604-4d4f-84c2-6ab8bb1124dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.520128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9d077365-9604-4d4f-84c2-6ab8bb1124dd None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:51.521695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9d077365-9604-4d4f-84c2-6ab8bb1124dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:51.557214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:51.563582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:51.592133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 482/974] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:51.593469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a484-ddd4-42f8-9cf5-a097bf29159f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.593758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a484-ddd4-42f8-9cf5-a097bf29159f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.594037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a484-ddd4-42f8-9cf5-a097bf29159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.603727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a3465-ef1c-458d-abe2-efaabb2d0287 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:51.610497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4056a484-ddd4-42f8-9cf5-a097bf29159f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:51.611994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4056a484-ddd4-42f8-9cf5-a097bf29159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:51.642292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcad0f-b133-46fe-a684-e3a339c5faef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:51.643108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcad0f-b133-46fe-a684-e3a339c5faef None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.643393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcad0f-b133-46fe-a684-e3a339c5faef None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.643670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcad0f-b133-46fe-a684-e3a339c5faef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.651780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcad0f-b133-46fe-a684-e3a339c5faef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:51.653263 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcad0f-b133-46fe-a684-e3a339c5faef None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-eebcad0f-b133-46fe-a684-e3a339c5faef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:50:51.668782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:51.669821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 493/975] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:50:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:51.670775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcad0f-b133-46fe-a684-e3a339c5faef None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:51.692103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 483/976] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:51.692997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056a484-ddd4-42f8-9cf5-a097bf29159f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:51.782342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcad0f-b133-46fe-a684-e3a339c5faef None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:51.802548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056a484-ddd4-42f8-9cf5-a097bf29159f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:51.802783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:51.806510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:51.821007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f37b-cbca-445e-9033-9fb1523dcb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:51.821751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f37b-cbca-445e-9033-9fb1523dcb62 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.822013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f37b-cbca-445e-9033-9fb1523dcb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.822305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f37b-cbca-445e-9033-9fb1523dcb62 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.823041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42f37b-cbca-445e-9033-9fb1523dcb62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:51.824433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42f37b-cbca-445e-9033-9fb1523dcb62 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:51.853120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 494/977] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:50:51 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:51.853120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f37b-cbca-445e-9033-9fb1523dcb62 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:51.855608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a618b-103a-4587-92b7-fb6a8449ca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:51.856434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a618b-103a-4587-92b7-fb6a8449ca4f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:51.856727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a618b-103a-4587-92b7-fb6a8449ca4f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:51.857009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a618b-103a-4587-92b7-fb6a8449ca4f None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:51.860079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a618b-103a-4587-92b7-fb6a8449ca4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:51.860079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a618b-103a-4587-92b7-fb6a8449ca4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:51.861853 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a618b-103a-4587-92b7-fb6a8449ca4f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:51.884262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 484/978] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jun 21 18:50:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:51.890041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a618b-103a-4587-92b7-fb6a8449ca4f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:51.974726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f37b-cbca-445e-9033-9fb1523dcb62 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.010734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a618b-103a-4587-92b7-fb6a8449ca4f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.012816 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2594d58-4912-4cc3-b1f9-4239513a8b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.014060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2594d58-4912-4cc3-b1f9-4239513a8b80 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.014060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2594d58-4912-4cc3-b1f9-4239513a8b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.014251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2594d58-4912-4cc3-b1f9-4239513a8b80 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.014884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2594d58-4912-4cc3-b1f9-4239513a8b80 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.016261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2594d58-4912-4cc3-b1f9-4239513a8b80 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f2594d58-4912-4cc3-b1f9-4239513a8b80 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:52.030105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:52.031133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:52.032000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 495/979] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:52.037368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2594d58-4912-4cc3-b1f9-4239513a8b80 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:52.049259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db79b08-c82f-4cff-a99b-e71d99a016e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.049907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db79b08-c82f-4cff-a99b-e71d99a016e2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.050191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db79b08-c82f-4cff-a99b-e71d99a016e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.050862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db79b08-c82f-4cff-a99b-e71d99a016e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.080399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:52.089245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:52.126114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db79b08-c82f-4cff-a99b-e71d99a016e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.127598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db79b08-c82f-4cff-a99b-e71d99a016e2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.152725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2594d58-4912-4cc3-b1f9-4239513a8b80 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.190394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f82306-19eb-46d9-8b79-22300b131245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.191060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f82306-19eb-46d9-8b79-22300b131245 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.191354 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f82306-19eb-46d9-8b79-22300b131245 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.191633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f82306-19eb-46d9-8b79-22300b131245 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.192337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f82306-19eb-46d9-8b79-22300b131245 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:52.192651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f82306-19eb-46d9-8b79-22300b131245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.194047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f82306-19eb-46d9-8b79-22300b131245 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.204523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 496/980] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:52.205427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f82306-19eb-46d9-8b79-22300b131245 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:52.232845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 485/981] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:50:51 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:50:52.238094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db79b08-c82f-4cff-a99b-e71d99a016e2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:52.295497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:52.311404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f82306-19eb-46d9-8b79-22300b131245 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.320848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:52.345130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db79b08-c82f-4cff-a99b-e71d99a016e2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.353629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e1de-6d2a-4b2f-b1bd-6de805b8d992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.354369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e1de-6d2a-4b2f-b1bd-6de805b8d992 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.354660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e1de-6d2a-4b2f-b1bd-6de805b8d992 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.354937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e1de-6d2a-4b2f-b1bd-6de805b8d992 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.355639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2e1de-6d2a-4b2f-b1bd-6de805b8d992 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:52.355952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2e1de-6d2a-4b2f-b1bd-6de805b8d992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.357333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2e1de-6d2a-4b2f-b1bd-6de805b8d992 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.363822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 497/982] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:52.369594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e1de-6d2a-4b2f-b1bd-6de805b8d992 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:52.388430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048ca3d-6335-4703-aa1a-5d842836f2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.389313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048ca3d-6335-4703-aa1a-5d842836f2ba None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.389651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048ca3d-6335-4703-aa1a-5d842836f2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.389935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048ca3d-6335-4703-aa1a-5d842836f2ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.400904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048ca3d-6335-4703-aa1a-5d842836f2ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.402381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048ca3d-6335-4703-aa1a-5d842836f2ba None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2048ca3d-6335-4703-aa1a-5d842836f2ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:50:52.420290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:52.421280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:52.422288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:50:52.422288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:50:52.422288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:50:52.422288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:52.422288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 486/983] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:52.427086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048ca3d-6335-4703-aa1a-5d842836f2ba None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:52.475113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e1de-6d2a-4b2f-b1bd-6de805b8d992 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.509148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7f886-b080-40e8-b38a-ef9841addc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.509760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7f886-b080-40e8-b38a-ef9841addc9b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.510064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7f886-b080-40e8-b38a-ef9841addc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.510344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7f886-b080-40e8-b38a-ef9841addc9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.511179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7f886-b080-40e8-b38a-ef9841addc9b None admin] RBAC: Adding query filter params (interface=admin, service_id=4afaacad8d4b4696b79caec3a97b8a17, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:52.511503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7f886-b080-40e8-b38a-ef9841addc9b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.512862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7f886-b080-40e8-b38a-ef9841addc9b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.522423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 498/984] 10.222.0.70 () {56 vars in 1249 bytes} [Tue Jun 21 18:50:52 2022] GET /identity/v3/endpoints?service_id=4afaacad8d4b4696b79caec3a97b8a17&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:52.523498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7f886-b080-40e8-b38a-ef9841addc9b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:52.540043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048ca3d-6335-4703-aa1a-5d842836f2ba None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.550992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:52.561464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:52.576645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aac1f-fb2d-491a-8a57-7d4e39990f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.577334 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aac1f-fb2d-491a-8a57-7d4e39990f5c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.577619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aac1f-fb2d-491a-8a57-7d4e39990f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.577897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aac1f-fb2d-491a-8a57-7d4e39990f5c None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.578612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aac1f-fb2d-491a-8a57-7d4e39990f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:52.578921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aac1f-fb2d-491a-8a57-7d4e39990f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.580284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aac1f-fb2d-491a-8a57-7d4e39990f5c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.592535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 487/985] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:52.597889 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aac1f-fb2d-491a-8a57-7d4e39990f5c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:52.635618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7f886-b080-40e8-b38a-ef9841addc9b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.670284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8096b-e3f6-4b84-8eda-429f0773f6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.671274 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8096b-e3f6-4b84-8eda-429f0773f6df None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.671763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8096b-e3f6-4b84-8eda-429f0773f6df None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.672257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8096b-e3f6-4b84-8eda-429f0773f6df None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.673172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8096b-e3f6-4b84-8eda-429f0773f6df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.674611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8096b-e3f6-4b84-8eda-429f0773f6df None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.681282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 499/986] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jun 21 18:50:52 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 18:50:52.705972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aac1f-fb2d-491a-8a57-7d4e39990f5c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.744766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732157b-6b0f-40d0-9998-0e2dff74bd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.745618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732157b-6b0f-40d0-9998-0e2dff74bd34 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.745904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732157b-6b0f-40d0-9998-0e2dff74bd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.746183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732157b-6b0f-40d0-9998-0e2dff74bd34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.754100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732157b-6b0f-40d0-9998-0e2dff74bd34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.755582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732157b-6b0f-40d0-9998-0e2dff74bd34 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3732157b-6b0f-40d0-9998-0e2dff74bd34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:50:52.771667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:52.772606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:52.779187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:50:52.779187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:50:52.779187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:50:52.779187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:50:52.779187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:50:52.779187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:50:52.779187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:50:52.779187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:50:52.779187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 488/987] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:52.779691 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8096b-e3f6-4b84-8eda-429f0773f6df None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:52.886216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:52.889669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8096b-e3f6-4b84-8eda-429f0773f6df None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:52.925822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455c51b-6b45-4969-8ec0-330618081314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:52.926487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455c51b-6b45-4969-8ec0-330618081314 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:52.926773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455c51b-6b45-4969-8ec0-330618081314 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:52.927057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455c51b-6b45-4969-8ec0-330618081314 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:52.927842 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455c51b-6b45-4969-8ec0-330618081314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:52.928153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455c51b-6b45-4969-8ec0-330618081314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:52.929610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455c51b-6b45-4969-8ec0-330618081314 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:52.942965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 500/988] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:50:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:52.949652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732157b-6b0f-40d0-9998-0e2dff74bd34 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:52.981660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:53.054006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732157b-6b0f-40d0-9998-0e2dff74bd34 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:53.088845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707c52a-27d1-408e-bb63-3bd02c13b475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:53.089761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707c52a-27d1-408e-bb63-3bd02c13b475 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:53.090032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707c52a-27d1-408e-bb63-3bd02c13b475 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:53.090288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707c52a-27d1-408e-bb63-3bd02c13b475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:53.091315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707c52a-27d1-408e-bb63-3bd02c13b475 None admin] RBAC: Adding query filter params (role.id=c87ba8b8c46b4328bde6b43b256e79e3, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=9fee9b6e04f24f96adb1a70306463a91) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:53.091615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707c52a-27d1-408e-bb63-3bd02c13b475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:53.093313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707c52a-27d1-408e-bb63-3bd02c13b475 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:53.105124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 489/989] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:50:52 2022] GET /identity/v3/role_assignments?user.id=9fee9b6e04f24f96adb1a70306463a91&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=c87ba8b8c46b4328bde6b43b256e79e3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:53.917050 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9f6d8-0c31-4be8-b402-2e0d9e218180 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:53.917623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9f6d8-0c31-4be8-b402-2e0d9e218180 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:53.917623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9f6d8-0c31-4be8-b402-2e0d9e218180 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:53.919151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 501/990] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:53.924092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fb06a-ef14-4c64-8391-a702ce16159a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:53.924389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fb06a-ef14-4c64-8391-a702ce16159a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:53.924674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fb06a-ef14-4c64-8391-a702ce16159a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:53.942177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-080fb06a-ef14-4c64-8391-a702ce16159a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:53.943366 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-080fb06a-ef14-4c64-8391-a702ce16159a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:54.013017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 490/991] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:54.038237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9f6d8-0c31-4be8-b402-2e0d9e218180 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:54.147852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9f6d8-0c31-4be8-b402-2e0d9e218180 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:54.183016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939326c-ee7d-4c7f-93a4-04ea523ecd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:54.183820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939326c-ee7d-4c7f-93a4-04ea523ecd87 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:54.184110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939326c-ee7d-4c7f-93a4-04ea523ecd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:54.184486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939326c-ee7d-4c7f-93a4-04ea523ecd87 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:54.185045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939326c-ee7d-4c7f-93a4-04ea523ecd87 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:54.186444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939326c-ee7d-4c7f-93a4-04ea523ecd87 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3939326c-ee7d-4c7f-93a4-04ea523ecd87 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:54.200110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:54.201141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:50:54.201988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:54.205172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 502/992] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:54.207266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fb06a-ef14-4c64-8391-a702ce16159a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:54.234371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:54.313538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fb06a-ef14-4c64-8391-a702ce16159a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:54.353332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42ce2f-0df4-4e34-8bd1-d51fee5089f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:54.354005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42ce2f-0df4-4e34-8bd1-d51fee5089f9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:54.354291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42ce2f-0df4-4e34-8bd1-d51fee5089f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:54.354571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42ce2f-0df4-4e34-8bd1-d51fee5089f9 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:54.355325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42ce2f-0df4-4e34-8bd1-d51fee5089f9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:54.355646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42ce2f-0df4-4e34-8bd1-d51fee5089f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:54.357025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42ce2f-0df4-4e34-8bd1-d51fee5089f9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:54.366547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 491/993] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:54.367755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939326c-ee7d-4c7f-93a4-04ea523ecd87 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:54.394407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:54.471517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939326c-ee7d-4c7f-93a4-04ea523ecd87 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:54.504977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437daf6-6a8f-4c19-84df-a99cb211fb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:54.505670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437daf6-6a8f-4c19-84df-a99cb211fb23 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:54.505963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437daf6-6a8f-4c19-84df-a99cb211fb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:54.506246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437daf6-6a8f-4c19-84df-a99cb211fb23 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:54.506950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437daf6-6a8f-4c19-84df-a99cb211fb23 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:54.507261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437daf6-6a8f-4c19-84df-a99cb211fb23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:54.509226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437daf6-6a8f-4c19-84df-a99cb211fb23 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:54.518894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 503/994] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:54.520075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42ce2f-0df4-4e34-8bd1-d51fee5089f9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:54.612652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:54.626550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42ce2f-0df4-4e34-8bd1-d51fee5089f9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:54.660847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c18c19-a2a5-4c6e-92bb-85ca26bb1e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:54.661539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c18c19-a2a5-4c6e-92bb-85ca26bb1e0f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:54.661828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c18c19-a2a5-4c6e-92bb-85ca26bb1e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:54.662110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c18c19-a2a5-4c6e-92bb-85ca26bb1e0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:54.662905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c18c19-a2a5-4c6e-92bb-85ca26bb1e0f None admin] RBAC: Adding query filter params (interface=public, service_id=bbbac8fda7ab4beca5c79964f3eaed86, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:54.663224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c18c19-a2a5-4c6e-92bb-85ca26bb1e0f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:54.664617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c18c19-a2a5-4c6e-92bb-85ca26bb1e0f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:54.674799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 492/995] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jun 21 18:50:54 2022] GET /identity/v3/endpoints?service_id=bbbac8fda7ab4beca5c79964f3eaed86&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:54.675984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437daf6-6a8f-4c19-84df-a99cb211fb23 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:54.755040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec75ec-d22c-4c24-ace6-006121f273bb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:54.755360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec75ec-d22c-4c24-ace6-006121f273bb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:54.755541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec75ec-d22c-4c24-ace6-006121f273bb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:54.756904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 493/996] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:54.761987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28226f8-ca4d-42e4-ba97-ddd0dbc377b4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:54.762279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28226f8-ca4d-42e4-ba97-ddd0dbc377b4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:54.762564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28226f8-ca4d-42e4-ba97-ddd0dbc377b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:54.771469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:54.779409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d28226f8-ca4d-42e4-ba97-ddd0dbc377b4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:54.780607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d28226f8-ca4d-42e4-ba97-ddd0dbc377b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:54.786086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437daf6-6a8f-4c19-84df-a99cb211fb23 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:54.822029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3f8c6-5ddf-438a-9914-75604a317bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:54.822990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f8c6-5ddf-438a-9914-75604a317bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:54.823458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f8c6-5ddf-438a-9914-75604a317bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:54.823915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f8c6-5ddf-438a-9914-75604a317bb0 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:54.824768 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3f8c6-5ddf-438a-9914-75604a317bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:54.826433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3f8c6-5ddf-438a-9914-75604a317bb0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:54.833384 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 504/997] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jun 21 18:50:54 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:50:54.853895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 494/998] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:54.878629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3f8c6-5ddf-438a-9914-75604a317bb0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:54.990117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3f8c6-5ddf-438a-9914-75604a317bb0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:55.024583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1da951-0803-4972-8795-2fcbdfb5567a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:55.025345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1da951-0803-4972-8795-2fcbdfb5567a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:55.025629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1da951-0803-4972-8795-2fcbdfb5567a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:55.025903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1da951-0803-4972-8795-2fcbdfb5567a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:55.026557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1da951-0803-4972-8795-2fcbdfb5567a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:55.027918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1da951-0803-4972-8795-2fcbdfb5567a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-cb1da951-0803-4972-8795-2fcbdfb5567a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:55.040622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:55.041844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:55.042745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 505/999] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jun 21 18:50:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:55.047998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28226f8-ca4d-42e4-ba97-ddd0dbc377b4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:55.130182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:55.149793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28226f8-ca4d-42e4-ba97-ddd0dbc377b4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:55.169199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3d5b8-f2c1-4165-9926-5fbc5cef145d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:55.169494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3d5b8-f2c1-4165-9926-5fbc5cef145d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:55.169770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3d5b8-f2c1-4165-9926-5fbc5cef145d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:55.171137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 506/1000] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:55.176165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c24646-bf4e-46a3-840a-be19951a00b5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:55.176459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c24646-bf4e-46a3-840a-be19951a00b5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:55.176747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c24646-bf4e-46a3-840a-be19951a00b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:55.186246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b6c84-ec3b-450a-b8b1-a55d5fa66f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:55.186915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b6c84-ec3b-450a-b8b1-a55d5fa66f6f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:55.187469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b6c84-ec3b-450a-b8b1-a55d5fa66f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:55.187748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b6c84-ec3b-450a-b8b1-a55d5fa66f6f None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:55.188893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b6c84-ec3b-450a-b8b1-a55d5fa66f6f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:55.189251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b6c84-ec3b-450a-b8b1-a55d5fa66f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:55.190641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b6c84-ec3b-450a-b8b1-a55d5fa66f6f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:55.193706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-86c24646-bf4e-46a3-840a-be19951a00b5 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:55.194900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-86c24646-bf4e-46a3-840a-be19951a00b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:55.195954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 495/1001] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:55.201476 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b6c84-ec3b-450a-b8b1-a55d5fa66f6f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:55.206718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:55.269245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 507/1002] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:55.292379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c24646-bf4e-46a3-840a-be19951a00b5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:55.311892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b6c84-ec3b-450a-b8b1-a55d5fa66f6f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:55.352126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac726280-5635-463b-b82b-ede9e31e9ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:55.353452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac726280-5635-463b-b82b-ede9e31e9ffb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:55.353743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac726280-5635-463b-b82b-ede9e31e9ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:55.354027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac726280-5635-463b-b82b-ede9e31e9ffb None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:55.354746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac726280-5635-463b-b82b-ede9e31e9ffb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:55.355061 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac726280-5635-463b-b82b-ede9e31e9ffb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:55.356449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac726280-5635-463b-b82b-ede9e31e9ffb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:55.366373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 496/1003] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jun 21 18:50:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:55.368169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac726280-5635-463b-b82b-ede9e31e9ffb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:55.380932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:55.409853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c24646-bf4e-46a3-840a-be19951a00b5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:55.445569 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64f2bd-0e82-4fa8-a5c9-df8d3ae54720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:55.446929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64f2bd-0e82-4fa8-a5c9-df8d3ae54720 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:55.447210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64f2bd-0e82-4fa8-a5c9-df8d3ae54720 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:55.447485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64f2bd-0e82-4fa8-a5c9-df8d3ae54720 None admin] PATH_INFO: `/v3/services/image` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:55.448139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64f2bd-0e82-4fa8-a5c9-df8d3ae54720 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:55.449540 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64f2bd-0e82-4fa8-a5c9-df8d3ae54720 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-4b64f2bd-0e82-4fa8-a5c9-df8d3ae54720 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:55.461262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:55.462385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:55.463245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 508/1004] 10.222.0.70 () {56 vars in 1105 bytes} [Tue Jun 21 18:50:55 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:50:55.468427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64f2bd-0e82-4fa8-a5c9-df8d3ae54720 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:55.480548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac726280-5635-463b-b82b-ede9e31e9ffb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:55.482833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:55.518774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c570ce-eb7a-4e38-8105-16a71e148ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:55.519445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c570ce-eb7a-4e38-8105-16a71e148ff7 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:55.519728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c570ce-eb7a-4e38-8105-16a71e148ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:55.520010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c570ce-eb7a-4e38-8105-16a71e148ff7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:55.520754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c570ce-eb7a-4e38-8105-16a71e148ff7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:55.522183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c570ce-eb7a-4e38-8105-16a71e148ff7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:55.545331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 497/1005] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jun 21 18:50:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:50:55.591480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64f2bd-0e82-4fa8-a5c9-df8d3ae54720 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:55.636584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f73710-1507-4366-a452-9b171bb2cdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:55.637298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f73710-1507-4366-a452-9b171bb2cdcf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:55.637729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f73710-1507-4366-a452-9b171bb2cdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:55.637848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f73710-1507-4366-a452-9b171bb2cdcf None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:55.638553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f73710-1507-4366-a452-9b171bb2cdcf None admin] RBAC: Adding query filter params (name=image) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:55.638871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f73710-1507-4366-a452-9b171bb2cdcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:55.640260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f73710-1507-4366-a452-9b171bb2cdcf None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:55.650371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 509/1006] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:50:55.651626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c570ce-eb7a-4e38-8105-16a71e148ff7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:55.729739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:55.769549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c570ce-eb7a-4e38-8105-16a71e148ff7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:55.804998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671d75b-3e31-4bc2-b793-7cd6732bef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:55.806158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d75b-3e31-4bc2-b793-7cd6732bef59 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:55.806450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d75b-3e31-4bc2-b793-7cd6732bef59 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:55.806723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d75b-3e31-4bc2-b793-7cd6732bef59 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:55.807432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671d75b-3e31-4bc2-b793-7cd6732bef59 None admin] RBAC: Adding query filter params (type=image) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:55.807741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671d75b-3e31-4bc2-b793-7cd6732bef59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:55.809124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671d75b-3e31-4bc2-b793-7cd6732bef59 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:55.821170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 498/1007] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:55 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:56.885554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe60f8-2523-4d4b-aefd-9ebee3b518ca None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:56.886066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe60f8-2523-4d4b-aefd-9ebee3b518ca None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:56.886144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe60f8-2523-4d4b-aefd-9ebee3b518ca None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:56.887501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 510/1008] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:56.892598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53fe18-11a9-49f1-9468-31c9a03b03d0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:56.893050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53fe18-11a9-49f1-9468-31c9a03b03d0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:56.893222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53fe18-11a9-49f1-9468-31c9a03b03d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:56.910600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6d53fe18-11a9-49f1-9468-31c9a03b03d0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:56.911915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6d53fe18-11a9-49f1-9468-31c9a03b03d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:56.982599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 499/1009] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:57.004793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe60f8-2523-4d4b-aefd-9ebee3b518ca None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:57.078332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:57.113625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe60f8-2523-4d4b-aefd-9ebee3b518ca None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:57.148862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6138d57-65c3-417f-9d12-788c5db62f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:57.150518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6138d57-65c3-417f-9d12-788c5db62f1c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:57.150809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6138d57-65c3-417f-9d12-788c5db62f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:57.151090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6138d57-65c3-417f-9d12-788c5db62f1c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:57.151745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6138d57-65c3-417f-9d12-788c5db62f1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:57.153136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6138d57-65c3-417f-9d12-788c5db62f1c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d6138d57-65c3-417f-9d12-788c5db62f1c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:50:57.165259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:50:57.166277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 18:50:57.167133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:50:57.168521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 511/1010] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:50:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:50:57.172337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53fe18-11a9-49f1-9468-31c9a03b03d0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:57.236739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:57.278274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53fe18-11a9-49f1-9468-31c9a03b03d0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:57.316421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3bfbc-1294-4349-ba9b-1050158a1a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:57.317148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3bfbc-1294-4349-ba9b-1050158a1a90 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:57.317436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3bfbc-1294-4349-ba9b-1050158a1a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:57.317714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3bfbc-1294-4349-ba9b-1050158a1a90 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:57.321123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3bfbc-1294-4349-ba9b-1050158a1a90 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:57.321123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3bfbc-1294-4349-ba9b-1050158a1a90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:57.321123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3bfbc-1294-4349-ba9b-1050158a1a90 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:57.329459 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 500/1011] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:57.335288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6138d57-65c3-417f-9d12-788c5db62f1c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:57.447452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6138d57-65c3-417f-9d12-788c5db62f1c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:57.485669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84c184-a5a7-4a37-a128-d22e699f698c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:57.486336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84c184-a5a7-4a37-a128-d22e699f698c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:57.486616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84c184-a5a7-4a37-a128-d22e699f698c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:57.486891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84c184-a5a7-4a37-a128-d22e699f698c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:57.487592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84c184-a5a7-4a37-a128-d22e699f698c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:50:57.487900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84c184-a5a7-4a37-a128-d22e699f698c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:57.489292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84c184-a5a7-4a37-a128-d22e699f698c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:57.490700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:57.501115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 512/1012] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jun 21 18:50:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:57.502615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3bfbc-1294-4349-ba9b-1050158a1a90 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:57.610031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3bfbc-1294-4349-ba9b-1050158a1a90 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:57.639762 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:57.649423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc1bc0-2f85-44cc-a0ad-6a1a07f9496a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:57.650099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc1bc0-2f85-44cc-a0ad-6a1a07f9496a None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:57.650389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc1bc0-2f85-44cc-a0ad-6a1a07f9496a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:57.650671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc1bc0-2f85-44cc-a0ad-6a1a07f9496a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:57.651434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc1bc0-2f85-44cc-a0ad-6a1a07f9496a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:57.652891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc1bc0-2f85-44cc-a0ad-6a1a07f9496a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:57.658081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996f1b4-a9b9-44f8-92bc-9edbe1cb4649 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:57.658381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996f1b4-a9b9-44f8-92bc-9edbe1cb4649 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:57.658664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996f1b4-a9b9-44f8-92bc-9edbe1cb4649 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:57.660312 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 513/1013] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:57.666686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcf9f-cc0c-46a3-a1f0-dabefbb9351a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:57.666985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcf9f-cc0c-46a3-a1f0-dabefbb9351a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:57.667269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcf9f-cc0c-46a3-a1f0-dabefbb9351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:57.674039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 501/1014] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jun 21 18:50:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:50:57.689991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0b1bcf9f-cc0c-46a3-a1f0-dabefbb9351a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:57.691245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0b1bcf9f-cc0c-46a3-a1f0-dabefbb9351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:57.775227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 514/1015] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:57 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:57.799180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc1bc0-2f85-44cc-a0ad-6a1a07f9496a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:57.865979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33639242-83c0-4cc7-9018-20d325d4e602 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:57.866176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33639242-83c0-4cc7-9018-20d325d4e602 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:57.866413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33639242-83c0-4cc7-9018-20d325d4e602 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:57.867855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 515/1016] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:50:57.873580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a4b4d-a4b9-4a77-892c-286897b71a15 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:57.873879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a4b4d-a4b9-4a77-892c-286897b71a15 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:57.874159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a4b4d-a4b9-4a77-892c-286897b71a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:57.893246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5b5a4b4d-a4b9-4a77-892c-286897b71a15 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:57.894616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5b5a4b4d-a4b9-4a77-892c-286897b71a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:57.919717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc1bc0-2f85-44cc-a0ad-6a1a07f9496a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:57.965355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaafed4-3ee7-4aa7-a468-1e2f0432fd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:57.966972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaafed4-3ee7-4aa7-a468-1e2f0432fd92 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:57.967501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaafed4-3ee7-4aa7-a468-1e2f0432fd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:57.968034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaafed4-3ee7-4aa7-a468-1e2f0432fd92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:57.969910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaafed4-3ee7-4aa7-a468-1e2f0432fd92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:57.971854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaafed4-3ee7-4aa7-a468-1e2f0432fd92 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:57.974347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 502/1017] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:50:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:50:57.979658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 516/1018] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:57 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:50:57.980819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaafed4-3ee7-4aa7-a468-1e2f0432fd92 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:58.002433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a4b4d-a4b9-4a77-892c-286897b71a15 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:50:58.043221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:58.069459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:50:58.100625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaafed4-3ee7-4aa7-a468-1e2f0432fd92 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:58.123373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a4b4d-a4b9-4a77-892c-286897b71a15 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:50:58.139458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9f438-895b-4ea1-816a-22aa5593ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:58.140130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9f438-895b-4ea1-816a-22aa5593ab77 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:58.140418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9f438-895b-4ea1-816a-22aa5593ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:58.140702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9f438-895b-4ea1-816a-22aa5593ab77 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:58.141483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9f438-895b-4ea1-816a-22aa5593ab77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:58.142954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9f438-895b-4ea1-816a-22aa5593ab77 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:58.161942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edded3-87c6-49d7-a05b-99440555eada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:50:58.162838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edded3-87c6-49d7-a05b-99440555eada None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:58.163132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edded3-87c6-49d7-a05b-99440555eada None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:58.163415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edded3-87c6-49d7-a05b-99440555eada None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:58.164174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edded3-87c6-49d7-a05b-99440555eada None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:50:58.165874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edded3-87c6-49d7-a05b-99440555eada None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:50:58.173776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 503/1019] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:50:57 2022] POST /identity/v3/projects => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:50:58.189131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 517/1020] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:50:58 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:50:59.848648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ac355-3702-4648-9e5a-c5fe22addc6a None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:59.849350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ac355-3702-4648-9e5a-c5fe22addc6a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:59.849350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ac355-3702-4648-9e5a-c5fe22addc6a None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:59.850801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 504/1021] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:50:59.855810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d206b-b8c5-4dd2-ad17-ab24849f80b8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:50:59.856223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d206b-b8c5-4dd2-ad17-ab24849f80b8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:50:59.856397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d206b-b8c5-4dd2-ad17-ab24849f80b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:50:59.873900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-415d206b-b8c5-4dd2-ad17-ab24849f80b8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:50:59.875106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-415d206b-b8c5-4dd2-ad17-ab24849f80b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:50:59.954213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 518/1022] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:50:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:50:59.978160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ac355-3702-4648-9e5a-c5fe22addc6a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.087930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ac355-3702-4648-9e5a-c5fe22addc6a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:00.090675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:00.125793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130cd8e-e7d2-4f53-89a8-bee80dd5785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:00.126524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130cd8e-e7d2-4f53-89a8-bee80dd5785f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.126812 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130cd8e-e7d2-4f53-89a8-bee80dd5785f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.127091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130cd8e-e7d2-4f53-89a8-bee80dd5785f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.137507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130cd8e-e7d2-4f53-89a8-bee80dd5785f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d130cd8e-e7d2-4f53-89a8-bee80dd5785f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:00.145716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:00.146659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:51:00.147500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:00.149460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 505/1023] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jun 21 18:50:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:00.152852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d206b-b8c5-4dd2-ad17-ab24849f80b8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.195199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:00.255570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d206b-b8c5-4dd2-ad17-ab24849f80b8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:00.264020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7103e-ff2c-4ca2-bfd3-3e8970beb82f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.264315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7103e-ff2c-4ca2-bfd3-3e8970beb82f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.264613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7103e-ff2c-4ca2-bfd3-3e8970beb82f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.266204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 506/1024] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:51:00.270910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f6aff-3c53-44fd-a365-8c04acf1ca44 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.271196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f6aff-3c53-44fd-a365-8c04acf1ca44 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.271478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f6aff-3c53-44fd-a365-8c04acf1ca44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.288517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e49f6aff-3c53-44fd-a365-8c04acf1ca44 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:00.290327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e49f6aff-3c53-44fd-a365-8c04acf1ca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:00.291846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82790596-0e78-41c0-bafd-cbde8ccd2b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:00.292523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82790596-0e78-41c0-bafd-cbde8ccd2b4f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.292808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82790596-0e78-41c0-bafd-cbde8ccd2b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.293113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82790596-0e78-41c0-bafd-cbde8ccd2b4f None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.293869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82790596-0e78-41c0-bafd-cbde8ccd2b4f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:00.294180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82790596-0e78-41c0-bafd-cbde8ccd2b4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:00.295771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82790596-0e78-41c0-bafd-cbde8ccd2b4f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:00.303858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 519/1025] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jun 21 18:51:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:00.305127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49723c6-bf59-4923-9ddb-f2ec9ce26095 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.305978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49723c6-bf59-4923-9ddb-f2ec9ce26095 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.306256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49723c6-bf59-4923-9ddb-f2ec9ce26095 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.307645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 520/1026] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:00.310097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49723c6-bf59-4923-9ddb-f2ec9ce26095 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.366044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 507/1027] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:51:00.367409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec6fc4-8367-4157-9525-8031f0d95d89 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.367697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec6fc4-8367-4157-9525-8031f0d95d89 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.368076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec6fc4-8367-4157-9525-8031f0d95d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.385016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a7ec6fc4-8367-4157-9525-8031f0d95d89 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:00.386174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a7ec6fc4-8367-4157-9525-8031f0d95d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:00.414065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49723c6-bf59-4923-9ddb-f2ec9ce26095 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:00.416228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:00.450368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25f8e8-fd97-409c-a057-1da6c37ab7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:00.451155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25f8e8-fd97-409c-a057-1da6c37ab7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.451446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25f8e8-fd97-409c-a057-1da6c37ab7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.451733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25f8e8-fd97-409c-a057-1da6c37ab7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.453671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 508/1028] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:00.454520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec6fc4-8367-4157-9525-8031f0d95d89 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.462882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25f8e8-fd97-409c-a057-1da6c37ab7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:00.464350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25f8e8-fd97-409c-a057-1da6c37ab7f7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:00.483440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7c25f8e8-fd97-409c-a057-1da6c37ab7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:51:00.484553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:00.485974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:00.486911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:51:00.486911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:51:00.486911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:00.486911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:00.486911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 521/1029] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:51:00.486911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25f8e8-fd97-409c-a057-1da6c37ab7f7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.561308 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec6fc4-8367-4157-9525-8031f0d95d89 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:00.584923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25f8e8-fd97-409c-a057-1da6c37ab7f7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:00.595107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5298c-cdd6-488d-bb46-b1f0c5ed0fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:00.595895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5298c-cdd6-488d-bb46-b1f0c5ed0fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.596188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5298c-cdd6-488d-bb46-b1f0c5ed0fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.596467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5298c-cdd6-488d-bb46-b1f0c5ed0fc1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.597162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5298c-cdd6-488d-bb46-b1f0c5ed0fc1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:00.598562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5298c-cdd6-488d-bb46-b1f0c5ed0fc1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-bbc5298c-cdd6-488d-bb46-b1f0c5ed0fc1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:00.610567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:00.611541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:00.612599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 509/1030] 10.222.0.70 () {56 vars in 1105 bytes} [Tue Jun 21 18:51:00 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:00.613813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5298c-cdd6-488d-bb46-b1f0c5ed0fc1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.622690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c95b3-7b5e-4862-92d4-244a5019e343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:00.623484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c95b3-7b5e-4862-92d4-244a5019e343 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.623773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c95b3-7b5e-4862-92d4-244a5019e343 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.624054 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c95b3-7b5e-4862-92d4-244a5019e343 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.625267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c95b3-7b5e-4862-92d4-244a5019e343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:00.626690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c95b3-7b5e-4862-92d4-244a5019e343 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:00.628445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 522/1031] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:00.629137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c95b3-7b5e-4862-92d4-244a5019e343 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.653924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:00.705045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:00.718366 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5298c-cdd6-488d-bb46-b1f0c5ed0fc1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:00.729747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c95b3-7b5e-4862-92d4-244a5019e343 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:00.751652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606711a1-67af-40a2-9132-35ec6fdcc036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:00.752319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606711a1-67af-40a2-9132-35ec6fdcc036 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.752602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606711a1-67af-40a2-9132-35ec6fdcc036 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.752877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606711a1-67af-40a2-9132-35ec6fdcc036 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.753631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606711a1-67af-40a2-9132-35ec6fdcc036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:00.753943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606711a1-67af-40a2-9132-35ec6fdcc036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:00.755383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606711a1-67af-40a2-9132-35ec6fdcc036 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:00.763988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233254-f699-4df4-897e-eba0f658c07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:00.764643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233254-f699-4df4-897e-eba0f658c07c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.764932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233254-f699-4df4-897e-eba0f658c07c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.765253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233254-f699-4df4-897e-eba0f658c07c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.765953 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233254-f699-4df4-897e-eba0f658c07c None admin] RBAC: Adding query filter params (name=image) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:00.766261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233254-f699-4df4-897e-eba0f658c07c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:00.767619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233254-f699-4df4-897e-eba0f658c07c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:00.768548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 510/1032] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:00.769201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606711a1-67af-40a2-9132-35ec6fdcc036 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.781123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 523/1033] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:00.781123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233254-f699-4df4-897e-eba0f658c07c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.864807 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:00.873427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606711a1-67af-40a2-9132-35ec6fdcc036 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:00.880455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233254-f699-4df4-897e-eba0f658c07c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:00.907922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168800aa-d843-431a-b85f-398ed082e792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:00.908514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168800aa-d843-431a-b85f-398ed082e792 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.908800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168800aa-d843-431a-b85f-398ed082e792 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.909078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168800aa-d843-431a-b85f-398ed082e792 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.909822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168800aa-d843-431a-b85f-398ed082e792 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:00.911254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168800aa-d843-431a-b85f-398ed082e792 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:00.915064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a830544-5d88-4da9-840e-d82cbfb2936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:00.915847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a830544-5d88-4da9-840e-d82cbfb2936e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:00.916135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a830544-5d88-4da9-840e-d82cbfb2936e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:00.916439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a830544-5d88-4da9-840e-d82cbfb2936e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:00.924254 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a830544-5d88-4da9-840e-d82cbfb2936e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:00.925795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a830544-5d88-4da9-840e-d82cbfb2936e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:00.932745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 511/1034] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:51:00 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:51:00.933595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168800aa-d843-431a-b85f-398ed082e792 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:00.938390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-2a830544-5d88-4da9-840e-d82cbfb2936e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:51:00.941642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:00.942678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:00.943627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 524/1035] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jun 21 18:51:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:00.949213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a830544-5d88-4da9-840e-d82cbfb2936e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:01.056868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168800aa-d843-431a-b85f-398ed082e792 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:01.062578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a830544-5d88-4da9-840e-d82cbfb2936e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:01.101276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd92c3a-959c-4c94-81aa-7a1ec0768d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:01.102022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd92c3a-959c-4c94-81aa-7a1ec0768d13 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:01.102825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd92c3a-959c-4c94-81aa-7a1ec0768d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:01.103155 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd92c3a-959c-4c94-81aa-7a1ec0768d13 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:01.103260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac1bfe-18d8-40a4-8c09-958d2a69c4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:01.103871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac1bfe-18d8-40a4-8c09-958d2a69c4eb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:01.104010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd92c3a-959c-4c94-81aa-7a1ec0768d13 None admin] RBAC: Adding query filter params (type=image) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:01.104185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac1bfe-18d8-40a4-8c09-958d2a69c4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:01.104281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd92c3a-959c-4c94-81aa-7a1ec0768d13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:01.104477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac1bfe-18d8-40a4-8c09-958d2a69c4eb None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:01.105308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac1bfe-18d8-40a4-8c09-958d2a69c4eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:01.105572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd92c3a-959c-4c94-81aa-7a1ec0768d13 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:01.105687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac1bfe-18d8-40a4-8c09-958d2a69c4eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:01.107041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac1bfe-18d8-40a4-8c09-958d2a69c4eb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:01.112191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 512/1036] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:01.118314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd92c3a-959c-4c94-81aa-7a1ec0768d13 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:01.127041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 525/1037] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jun 21 18:51:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:01.132951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac1bfe-18d8-40a4-8c09-958d2a69c4eb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:01.137727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:01.198651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:01.236453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd92c3a-959c-4c94-81aa-7a1ec0768d13 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:01.251936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac1bfe-18d8-40a4-8c09-958d2a69c4eb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:01.272881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d66745-d9c8-4c66-b799-ceee2fbaa8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:01.273756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d66745-d9c8-4c66-b799-ceee2fbaa8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:01.274629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d66745-d9c8-4c66-b799-ceee2fbaa8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:01.274972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d66745-d9c8-4c66-b799-ceee2fbaa8c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:01.276002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d66745-d9c8-4c66-b799-ceee2fbaa8c2 None admin] RBAC: Adding query filter params (interface=public, service_id=5e9ede565a5749e4a1f8d95235875d8f, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:01.276382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d66745-d9c8-4c66-b799-ceee2fbaa8c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:01.278054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d66745-d9c8-4c66-b799-ceee2fbaa8c2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:01.284024 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 513/1038] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jun 21 18:51:01 2022] GET /identity/v3/endpoints?service_id=5e9ede565a5749e4a1f8d95235875d8f&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:01.289304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d66745-d9c8-4c66-b799-ceee2fbaa8c2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:01.291393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d36c1-eea8-4a80-b9ae-90fe2d183af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:01.292107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d36c1-eea8-4a80-b9ae-90fe2d183af6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:01.292457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d36c1-eea8-4a80-b9ae-90fe2d183af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:01.292901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d36c1-eea8-4a80-b9ae-90fe2d183af6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:01.377239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d36c1-eea8-4a80-b9ae-90fe2d183af6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:01.378932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d36c1-eea8-4a80-b9ae-90fe2d183af6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:01.382515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:01.402357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d66745-d9c8-4c66-b799-ceee2fbaa8c2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:01.434670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:01.442519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c821d5-2ebf-4cb2-8b5c-bc6a95b864d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:01.443830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c821d5-2ebf-4cb2-8b5c-bc6a95b864d8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:01.444541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c821d5-2ebf-4cb2-8b5c-bc6a95b864d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:01.445018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c821d5-2ebf-4cb2-8b5c-bc6a95b864d8 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:01.446747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c821d5-2ebf-4cb2-8b5c-bc6a95b864d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:01.449064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c821d5-2ebf-4cb2-8b5c-bc6a95b864d8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:01.460362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 514/1039] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jun 21 18:51:01 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:51:01.506454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 526/1040] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:01 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:51:01.507782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c821d5-2ebf-4cb2-8b5c-bc6a95b864d8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:01.624052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c821d5-2ebf-4cb2-8b5c-bc6a95b864d8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:01.660330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f7fd7-80cd-4447-95d0-da62cf67fabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:01.661164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f7fd7-80cd-4447-95d0-da62cf67fabc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:01.661458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f7fd7-80cd-4447-95d0-da62cf67fabc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:01.661743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f7fd7-80cd-4447-95d0-da62cf67fabc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:01.672912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f7fd7-80cd-4447-95d0-da62cf67fabc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:01.674395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f7fd7-80cd-4447-95d0-da62cf67fabc None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-778f7fd7-80cd-4447-95d0-da62cf67fabc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:51:01.692086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:01.693068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:01.694303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:51:01.694303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:51:01.694303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:01.694303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:01.694303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 515/1041] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:01.699855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d36c1-eea8-4a80-b9ae-90fe2d183af6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:01.804122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d36c1-eea8-4a80-b9ae-90fe2d183af6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:01.807278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:01.842065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27453a9-fa7c-4b49-a3dc-37d1444313b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:01.842875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27453a9-fa7c-4b49-a3dc-37d1444313b3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:01.843268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27453a9-fa7c-4b49-a3dc-37d1444313b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:01.843649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27453a9-fa7c-4b49-a3dc-37d1444313b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:01.844473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27453a9-fa7c-4b49-a3dc-37d1444313b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:01.844890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27453a9-fa7c-4b49-a3dc-37d1444313b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:01.846380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27453a9-fa7c-4b49-a3dc-37d1444313b3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:01.862887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 527/1042] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:01.864090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f7fd7-80cd-4447-95d0-da62cf67fabc None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:01.915827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:01.974248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f7fd7-80cd-4447-95d0-da62cf67fabc None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:02.008359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef54d7-865c-41b2-9598-b2fde15e37a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:02.009175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef54d7-865c-41b2-9598-b2fde15e37a9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:02.009468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef54d7-865c-41b2-9598-b2fde15e37a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:02.009748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef54d7-865c-41b2-9598-b2fde15e37a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:02.016619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef54d7-865c-41b2-9598-b2fde15e37a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:02.018157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef54d7-865c-41b2-9598-b2fde15e37a9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-32ef54d7-865c-41b2-9598-b2fde15e37a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:51:02.033593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:02.034705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:02.035852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:51:02.035852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:51:02.035852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:51:02.035852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:51:02.035852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:51:02.035852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:51:02.035852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:51:02.035852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:02.037163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 516/1043] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:02.041487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27453a9-fa7c-4b49-a3dc-37d1444313b3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:02.149924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27453a9-fa7c-4b49-a3dc-37d1444313b3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:02.184590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0339b72-1915-4629-8853-1eef5e4cc0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:02.185283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339b72-1915-4629-8853-1eef5e4cc0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:02.185576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339b72-1915-4629-8853-1eef5e4cc0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:02.185859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339b72-1915-4629-8853-1eef5e4cc0f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:02.186649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339b72-1915-4629-8853-1eef5e4cc0f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:02.187214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339b72-1915-4629-8853-1eef5e4cc0f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:02.188648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339b72-1915-4629-8853-1eef5e4cc0f4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:02.201555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 528/1044] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:02.207305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef54d7-865c-41b2-9598-b2fde15e37a9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:02.308530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef54d7-865c-41b2-9598-b2fde15e37a9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:02.321778 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:02.353061 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7e0ea-63ec-406c-87bc-5667cf998b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:02.353819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7e0ea-63ec-406c-87bc-5667cf998b77 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:02.354178 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7e0ea-63ec-406c-87bc-5667cf998b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:02.354531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7e0ea-63ec-406c-87bc-5667cf998b77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:02.355428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7e0ea-63ec-406c-87bc-5667cf998b77 None admin] RBAC: Adding query filter params (role.id=c87ba8b8c46b4328bde6b43b256e79e3, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=dcbcec65220c4ebb95f974925fa73f34) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:02.355817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7e0ea-63ec-406c-87bc-5667cf998b77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:02.357368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7e0ea-63ec-406c-87bc-5667cf998b77 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:02.363197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 517/1045] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:02 2022] GET /identity/v3/role_assignments?user.id=dcbcec65220c4ebb95f974925fa73f34&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=c87ba8b8c46b4328bde6b43b256e79e3 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:03.040577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8e25-6870-4fc6-9f14-ec19da630cba None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.040978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8e25-6870-4fc6-9f14-ec19da630cba None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.041192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8e25-6870-4fc6-9f14-ec19da630cba None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.042542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 529/1046] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:03.048460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b8514-5dc0-44d0-8f92-634bf5cf41a1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.048860 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b8514-5dc0-44d0-8f92-634bf5cf41a1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.049052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b8514-5dc0-44d0-8f92-634bf5cf41a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.066461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9d1b8514-5dc0-44d0-8f92-634bf5cf41a1 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:03.067656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9d1b8514-5dc0-44d0-8f92-634bf5cf41a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:03.136219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 518/1047] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:03.159873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8e25-6870-4fc6-9f14-ec19da630cba None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:03.186095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:03.262870 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8e25-6870-4fc6-9f14-ec19da630cba None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:03.296992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06657974-58b4-4c38-9241-613458576cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:03.297727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06657974-58b4-4c38-9241-613458576cbe None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.298009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06657974-58b4-4c38-9241-613458576cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.298284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06657974-58b4-4c38-9241-613458576cbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.309026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06657974-58b4-4c38-9241-613458576cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-06657974-58b4-4c38-9241-613458576cbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:03.316884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:03.317813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:03.318706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:03.329140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 530/1048] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:51:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:03.329300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b8514-5dc0-44d0-8f92-634bf5cf41a1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:03.429534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b8514-5dc0-44d0-8f92-634bf5cf41a1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:03.431105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:03.467352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343419da-7698-4e1a-b4cc-3722e046d271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:03.468017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343419da-7698-4e1a-b4cc-3722e046d271 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.468299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343419da-7698-4e1a-b4cc-3722e046d271 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.468578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343419da-7698-4e1a-b4cc-3722e046d271 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.469348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343419da-7698-4e1a-b4cc-3722e046d271 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:03.469658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343419da-7698-4e1a-b4cc-3722e046d271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:03.471020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343419da-7698-4e1a-b4cc-3722e046d271 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:03.483977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 519/1049] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:03.485890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06657974-58b4-4c38-9241-613458576cbe None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:03.524081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49518d-076a-41b0-b889-98fa0a8b98fd None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.524378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49518d-076a-41b0-b889-98fa0a8b98fd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.524665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49518d-076a-41b0-b889-98fa0a8b98fd None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.526223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 520/1050] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:03.530949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cea01-85a0-410e-83e4-47551c2fbf4a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.531251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cea01-85a0-410e-83e4-47551c2fbf4a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.531533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cea01-85a0-410e-83e4-47551c2fbf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.548954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e41cea01-85a0-410e-83e4-47551c2fbf4a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:03.550240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e41cea01-85a0-410e-83e4-47551c2fbf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:03.576704 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:03.594453 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06657974-58b4-4c38-9241-613458576cbe None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:03.626353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 521/1051] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:51:03.633005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72081588-d655-4f34-a69f-86ebff728f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:03.634441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72081588-d655-4f34-a69f-86ebff728f0a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.634723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72081588-d655-4f34-a69f-86ebff728f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.634996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72081588-d655-4f34-a69f-86ebff728f0a None admin] PATH_INFO: `/v3/users/ac00373fb8c040dfbb033f0bab0274f2` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.648964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72081588-d655-4f34-a69f-86ebff728f0a None admin] RBAC: Authorizing `identity:get_user(user_id=ac00373fb8c040dfbb033f0bab0274f2)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:03.649737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cea01-85a0-410e-83e4-47551c2fbf4a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:03.650445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72081588-d655-4f34-a69f-86ebff728f0a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:03.656617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 531/1052] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:03 2022] GET /identity/v3/users/ac00373fb8c040dfbb033f0bab0274f2 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:03.658663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72081588-d655-4f34-a69f-86ebff728f0a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:03.758942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:03.765677 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cea01-85a0-410e-83e4-47551c2fbf4a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:03.766325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72081588-d655-4f34-a69f-86ebff728f0a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:03.803293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0127f-e3e9-42d0-805c-f655ba314749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:03.804004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0127f-e3e9-42d0-805c-f655ba314749 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.804284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0127f-e3e9-42d0-805c-f655ba314749 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.804380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ca59e-676a-40d9-9d0f-ffd90899d2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:03.804559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0127f-e3e9-42d0-805c-f655ba314749 None admin] PATH_INFO: `/v3/services/image` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.805830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ca59e-676a-40d9-9d0f-ffd90899d2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.805932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0127f-e3e9-42d0-805c-f655ba314749 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:03.806285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ca59e-676a-40d9-9d0f-ffd90899d2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.806664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ca59e-676a-40d9-9d0f-ffd90899d2b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.807234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0127f-e3e9-42d0-805c-f655ba314749 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-21e0127f-e3e9-42d0-805c-f655ba314749 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:03.819419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:03.820635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:03.822040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 522/1053] 10.222.0.70 () {56 vars in 1105 bytes} [Tue Jun 21 18:51:03 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:51:03.825112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:03.827255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0127f-e3e9-42d0-805c-f655ba314749 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:03.880390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ca59e-676a-40d9-9d0f-ffd90899d2b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:03.881866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ca59e-676a-40d9-9d0f-ffd90899d2b7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:03.939312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0127f-e3e9-42d0-805c-f655ba314749 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:03.975673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87191395-a45c-463c-adf9-55ec1b31bd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:03.976502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87191395-a45c-463c-adf9-55ec1b31bd3b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:03.977319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87191395-a45c-463c-adf9-55ec1b31bd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:03.977615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87191395-a45c-463c-adf9-55ec1b31bd3b None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:03.978333 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87191395-a45c-463c-adf9-55ec1b31bd3b None admin] RBAC: Adding query filter params (name=image) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:03.978642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87191395-a45c-463c-adf9-55ec1b31bd3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:03.980021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87191395-a45c-463c-adf9-55ec1b31bd3b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:03.989118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 523/1054] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:03.991081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87191395-a45c-463c-adf9-55ec1b31bd3b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:03.998712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 532/1055] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:51:04.004336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ca59e-676a-40d9-9d0f-ffd90899d2b7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:04.019893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:04.051511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:04.111461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ca59e-676a-40d9-9d0f-ffd90899d2b7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:04.111641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87191395-a45c-463c-adf9-55ec1b31bd3b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:04.148324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e818dc-502c-4dc4-a19a-5e2e375a2042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:04.149243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e818dc-502c-4dc4-a19a-5e2e375a2042 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:04.149528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e818dc-502c-4dc4-a19a-5e2e375a2042 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:04.149804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e818dc-502c-4dc4-a19a-5e2e375a2042 None admin] PATH_INFO: `/v3/projects/a7d9e58343ff486abea5e6eb7375de83` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:04.150959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e818dc-502c-4dc4-a19a-5e2e375a2042 None admin] RBAC: Authorizing `identity:get_project(project_id=a7d9e58343ff486abea5e6eb7375de83)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:04.151554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092120c-4c00-451d-9fd1-f4cb577df404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:04.152216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092120c-4c00-451d-9fd1-f4cb577df404 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:04.152396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e818dc-502c-4dc4-a19a-5e2e375a2042 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:04.152560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092120c-4c00-451d-9fd1-f4cb577df404 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:04.152780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092120c-4c00-451d-9fd1-f4cb577df404 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:04.153517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092120c-4c00-451d-9fd1-f4cb577df404 None admin] RBAC: Adding query filter params (type=image) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:04.153828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092120c-4c00-451d-9fd1-f4cb577df404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:04.154375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 533/1056] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:04 2022] GET /identity/v3/projects/a7d9e58343ff486abea5e6eb7375de83 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:04.158277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092120c-4c00-451d-9fd1-f4cb577df404 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:04.160662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e818dc-502c-4dc4-a19a-5e2e375a2042 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:04.163638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 524/1057] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:04.168915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092120c-4c00-451d-9fd1-f4cb577df404 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:04.264775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:04.279357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e818dc-502c-4dc4-a19a-5e2e375a2042 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:04.280993 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092120c-4c00-451d-9fd1-f4cb577df404 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:04.291052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:04.316108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58472eb-beb0-48e1-8196-f225fac9678c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:04.316760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58472eb-beb0-48e1-8196-f225fac9678c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:04.317044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58472eb-beb0-48e1-8196-f225fac9678c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:04.317354 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58472eb-beb0-48e1-8196-f225fac9678c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:04.321311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58472eb-beb0-48e1-8196-f225fac9678c None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=a7d9e58343ff486abea5e6eb7375de83, user.id=ac00373fb8c040dfbb033f0bab0274f2) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:04.321311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58472eb-beb0-48e1-8196-f225fac9678c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:04.321494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db7e1e-1dd6-4b77-9b59-05761df1c366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:04.321603 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58472eb-beb0-48e1-8196-f225fac9678c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:04.321678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db7e1e-1dd6-4b77-9b59-05761df1c366 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:04.321678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db7e1e-1dd6-4b77-9b59-05761df1c366 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:04.321678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db7e1e-1dd6-4b77-9b59-05761df1c366 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:04.321855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db7e1e-1dd6-4b77-9b59-05761df1c366 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:04.323990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db7e1e-1dd6-4b77-9b59-05761df1c366 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:04.337284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 534/1058] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:04 2022] GET /identity/v3/role_assignments?user.id=ac00373fb8c040dfbb033f0bab0274f2&scope.project.id=a7d9e58343ff486abea5e6eb7375de83&role.id=e089a0d6040040c4bcf143a680488e3d => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:04.346006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 525/1059] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jun 21 18:51:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:51:04.386365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe79593-3284-4e6b-a120-101fe25b0ce9 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:04.386579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe79593-3284-4e6b-a120-101fe25b0ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:04.386849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe79593-3284-4e6b-a120-101fe25b0ce9 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:04.388568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 535/1060] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:04.394492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1a167-0fe2-45ca-b8c4-92357faf8416 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:04.394792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1a167-0fe2-45ca-b8c4-92357faf8416 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:04.395074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1a167-0fe2-45ca-b8c4-92357faf8416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:04.413776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d0c1a167-0fe2-45ca-b8c4-92357faf8416 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:04.415783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d0c1a167-0fe2-45ca-b8c4-92357faf8416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:04.510753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 526/1061] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:04.532685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe79593-3284-4e6b-a120-101fe25b0ce9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:04.642950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe79593-3284-4e6b-a120-101fe25b0ce9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:04.678364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cec56f-9543-4ce7-be37-42ae6339e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:04.679188 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cec56f-9543-4ce7-be37-42ae6339e209 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:04.679475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cec56f-9543-4ce7-be37-42ae6339e209 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:04.679755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cec56f-9543-4ce7-be37-42ae6339e209 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:04.691475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cec56f-9543-4ce7-be37-42ae6339e209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a9cec56f-9543-4ce7-be37-42ae6339e209 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:04.699864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:04.700801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:04.702024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 536/1062] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jun 21 18:51:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:04.709679 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1a167-0fe2-45ca-b8c4-92357faf8416 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:04.777199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:04.815245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1a167-0fe2-45ca-b8c4-92357faf8416 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:04.851032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f7c62-ba63-48e4-98c4-22cf58b9cc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:04.852429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f7c62-ba63-48e4-98c4-22cf58b9cc16 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:04.852828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f7c62-ba63-48e4-98c4-22cf58b9cc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:04.853246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f7c62-ba63-48e4-98c4-22cf58b9cc16 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:04.854106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f7c62-ba63-48e4-98c4-22cf58b9cc16 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:04.854546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f7c62-ba63-48e4-98c4-22cf58b9cc16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:04.856024 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f7c62-ba63-48e4-98c4-22cf58b9cc16 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:04.866737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 527/1063] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jun 21 18:51:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:04.872548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cec56f-9543-4ce7-be37-42ae6339e209 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:04.909146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:04.978452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cec56f-9543-4ce7-be37-42ae6339e209 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:05.011855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e69f3c-c7ea-48a8-aa21-bdf6bd70c4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:05.012649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e69f3c-c7ea-48a8-aa21-bdf6bd70c4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:05.012935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e69f3c-c7ea-48a8-aa21-bdf6bd70c4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:05.013241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e69f3c-c7ea-48a8-aa21-bdf6bd70c4b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:05.024456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e69f3c-c7ea-48a8-aa21-bdf6bd70c4b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:05.025935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e69f3c-c7ea-48a8-aa21-bdf6bd70c4b2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b9e69f3c-c7ea-48a8-aa21-bdf6bd70c4b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:51:05.043999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:05.044961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:05.045920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:51:05.045920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:51:05.045920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:05.045920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:05.049410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 537/1064] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:05.050968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f7c62-ba63-48e4-98c4-22cf58b9cc16 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:05.157355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f7c62-ba63-48e4-98c4-22cf58b9cc16 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:05.186565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:05.194103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a711b8-9f07-48b0-bb5c-6cfa2b931c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:05.194749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a711b8-9f07-48b0-bb5c-6cfa2b931c25 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:05.195039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a711b8-9f07-48b0-bb5c-6cfa2b931c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:05.195321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a711b8-9f07-48b0-bb5c-6cfa2b931c25 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:05.196651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a711b8-9f07-48b0-bb5c-6cfa2b931c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:05.196966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a711b8-9f07-48b0-bb5c-6cfa2b931c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:05.198381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a711b8-9f07-48b0-bb5c-6cfa2b931c25 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:05.210342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 528/1065] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:05.216333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e69f3c-c7ea-48a8-aa21-bdf6bd70c4b2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:05.301971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:05.325219 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e69f3c-c7ea-48a8-aa21-bdf6bd70c4b2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:05.361720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c82d2a-e102-4d3c-b72b-4274cb6e15a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:05.362512 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c82d2a-e102-4d3c-b72b-4274cb6e15a5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:05.362800 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c82d2a-e102-4d3c-b72b-4274cb6e15a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:05.363084 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c82d2a-e102-4d3c-b72b-4274cb6e15a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:05.370297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c82d2a-e102-4d3c-b72b-4274cb6e15a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:05.371729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c82d2a-e102-4d3c-b72b-4274cb6e15a5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d7c82d2a-e102-4d3c-b72b-4274cb6e15a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:51:05.386639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:05.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:51:05.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:05.389194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 538/1066] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jun 21 18:51:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:05.393565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a711b8-9f07-48b0-bb5c-6cfa2b931c25 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:05.499856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a711b8-9f07-48b0-bb5c-6cfa2b931c25 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:05.535107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a6324-5c87-4196-a36e-509e5b68eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:05.535790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a6324-5c87-4196-a36e-509e5b68eb10 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:05.536069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a6324-5c87-4196-a36e-509e5b68eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:05.536351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a6324-5c87-4196-a36e-509e5b68eb10 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:05.537229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a6324-5c87-4196-a36e-509e5b68eb10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:05.537543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a6324-5c87-4196-a36e-509e5b68eb10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:05.539020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a6324-5c87-4196-a36e-509e5b68eb10 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:05.558254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 529/1067] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jun 21 18:51:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:05.564028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c82d2a-e102-4d3c-b72b-4274cb6e15a5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:05.674010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c82d2a-e102-4d3c-b72b-4274cb6e15a5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:05.707605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d3567-d7c0-4f6c-879b-e6369eba1444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:05.708390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3567-d7c0-4f6c-879b-e6369eba1444 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:05.708677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3567-d7c0-4f6c-879b-e6369eba1444 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:05.708956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3567-d7c0-4f6c-879b-e6369eba1444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:05.719665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d3567-d7c0-4f6c-879b-e6369eba1444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:05.721116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d3567-d7c0-4f6c-879b-e6369eba1444 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:05.728434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-067d3567-d7c0-4f6c-879b-e6369eba1444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:51:05.741170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:05.742120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:05.747676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:51:05.747676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:51:05.747676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:05.747676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:05.747676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 539/1068] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:51:05.748448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a6324-5c87-4196-a36e-509e5b68eb10 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:05.798431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:05.850153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a6324-5c87-4196-a36e-509e5b68eb10 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:05.884348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144165f0-c791-4d54-8fe7-60d1c44a4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:05.885011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144165f0-c791-4d54-8fe7-60d1c44a4e79 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:05.885325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144165f0-c791-4d54-8fe7-60d1c44a4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:05.885613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144165f0-c791-4d54-8fe7-60d1c44a4e79 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:05.886340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144165f0-c791-4d54-8fe7-60d1c44a4e79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:05.886655 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144165f0-c791-4d54-8fe7-60d1c44a4e79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:05.888044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144165f0-c791-4d54-8fe7-60d1c44a4e79 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:05.904503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 530/1069] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:05.905669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d3567-d7c0-4f6c-879b-e6369eba1444 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:06.007751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d3567-d7c0-4f6c-879b-e6369eba1444 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:06.043012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09366a27-14c0-4582-b624-3e104aca080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:06.043812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09366a27-14c0-4582-b624-3e104aca080b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.044100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09366a27-14c0-4582-b624-3e104aca080b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.044382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09366a27-14c0-4582-b624-3e104aca080b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.051386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09366a27-14c0-4582-b624-3e104aca080b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:06.052772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09366a27-14c0-4582-b624-3e104aca080b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-09366a27-14c0-4582-b624-3e104aca080b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:51:06.067868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:06.068809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:06.074655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:51:06.074655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:51:06.074655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:51:06.074655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:51:06.074655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:51:06.074655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:51:06.074655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:51:06.074655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:06.074655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 540/1070] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:06.076861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144165f0-c791-4d54-8fe7-60d1c44a4e79 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:06.181808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144165f0-c791-4d54-8fe7-60d1c44a4e79 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:06.191580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:06.218036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6af9f6-f064-494f-adbd-fc42ff026c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:06.218717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6af9f6-f064-494f-adbd-fc42ff026c5f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.219006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6af9f6-f064-494f-adbd-fc42ff026c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.219286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6af9f6-f064-494f-adbd-fc42ff026c5f None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.220093 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6af9f6-f064-494f-adbd-fc42ff026c5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:06.220411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6af9f6-f064-494f-adbd-fc42ff026c5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:06.221935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6af9f6-f064-494f-adbd-fc42ff026c5f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:06.240127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 531/1071] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:51:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:06.241341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09366a27-14c0-4582-b624-3e104aca080b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:06.306189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:06.360164 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09366a27-14c0-4582-b624-3e104aca080b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:06.369495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bea27-799f-4c75-97be-f7edb6c27c17 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.369796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bea27-799f-4c75-97be-f7edb6c27c17 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.370078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bea27-799f-4c75-97be-f7edb6c27c17 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.372635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 532/1072] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:06.376783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ab5e7-01f1-4c5e-8cf1-88b3f494ecf6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.377079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ab5e7-01f1-4c5e-8cf1-88b3f494ecf6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.377404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ab5e7-01f1-4c5e-8cf1-88b3f494ecf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.394646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0f2ab5e7-01f1-4c5e-8cf1-88b3f494ecf6 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:06.395852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0f2ab5e7-01f1-4c5e-8cf1-88b3f494ecf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:06.396905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282fab1-9c26-4e4f-92ff-6371d0f85bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:06.397944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282fab1-9c26-4e4f-92ff-6371d0f85bfe None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.398226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282fab1-9c26-4e4f-92ff-6371d0f85bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.398524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282fab1-9c26-4e4f-92ff-6371d0f85bfe None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/dcbcec65220c4ebb95f974925fa73f34/roles/c87ba8b8c46b4328bde6b43b256e79e3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.401409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282fab1-9c26-4e4f-92ff-6371d0f85bfe None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=dcbcec65220c4ebb95f974925fa73f34, role_id=c87ba8b8c46b4328bde6b43b256e79e3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:06.402848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282fab1-9c26-4e4f-92ff-6371d0f85bfe None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:06.418552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 541/1073] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:51:06 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/dcbcec65220c4ebb95f974925fa73f34/roles/c87ba8b8c46b4328bde6b43b256e79e3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:51:06.477552 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 533/1074] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:51:06.501333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282fab1-9c26-4e4f-92ff-6371d0f85bfe None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:06.506330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510910-a6e1-4535-bc61-0ee1140af992 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.506671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510910-a6e1-4535-bc61-0ee1140af992 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.506915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510910-a6e1-4535-bc61-0ee1140af992 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.508554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 534/1075] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:51:06.514992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8e6b3-d48a-4457-818e-8302b407bcb7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.515300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8e6b3-d48a-4457-818e-8302b407bcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.515572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8e6b3-d48a-4457-818e-8302b407bcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.534594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8fe8e6b3-d48a-4457-818e-8302b407bcb7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:06.535846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8fe8e6b3-d48a-4457-818e-8302b407bcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:06.619007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282fab1-9c26-4e4f-92ff-6371d0f85bfe None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:06.624285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 535/1076] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:51:06.640147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:06.647790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8e6b3-d48a-4457-818e-8302b407bcb7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:06.661417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cbd22-5d71-4bb0-9d5e-f0b92518bea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:06.662706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cbd22-5d71-4bb0-9d5e-f0b92518bea0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.663178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cbd22-5d71-4bb0-9d5e-f0b92518bea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.664002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cbd22-5d71-4bb0-9d5e-f0b92518bea0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.677278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:06.680541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7cbd22-5d71-4bb0-9d5e-f0b92518bea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-6f7cbd22-5d71-4bb0-9d5e-f0b92518bea0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:06.688621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:06.689806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:06.690653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:06.693233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 542/1077] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:06.696786 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cbd22-5d71-4bb0-9d5e-f0b92518bea0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:06.759879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8e6b3-d48a-4457-818e-8302b407bcb7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:06.796891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ca67-3857-46bc-b7f3-1930f2994567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:06.797570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ca67-3857-46bc-b7f3-1930f2994567 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.797858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ca67-3857-46bc-b7f3-1930f2994567 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.798126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ca67-3857-46bc-b7f3-1930f2994567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.804477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cbd22-5d71-4bb0-9d5e-f0b92518bea0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:06.843284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7993fd-9e3f-4778-ab3c-dce9e6db1f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:06.844046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7993fd-9e3f-4778-ab3c-dce9e6db1f28 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:06.844402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7993fd-9e3f-4778-ab3c-dce9e6db1f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:06.844689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7993fd-9e3f-4778-ab3c-dce9e6db1f28 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:06.845515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7993fd-9e3f-4778-ab3c-dce9e6db1f28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:06.845874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7993fd-9e3f-4778-ab3c-dce9e6db1f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:06.847239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7993fd-9e3f-4778-ab3c-dce9e6db1f28 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:06.856476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 543/1078] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:06.862191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7993fd-9e3f-4778-ab3c-dce9e6db1f28 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:06.877413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7ca67-3857-46bc-b7f3-1930f2994567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:06.879043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7ca67-3857-46bc-b7f3-1930f2994567 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:06.905871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:06.912936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:06.971765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7993fd-9e3f-4778-ab3c-dce9e6db1f28 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:06.998014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 536/1079] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:51:07.004006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ca67-3857-46bc-b7f3-1930f2994567 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:07.010902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d91426-d164-4de6-94e6-e81f6ee9e76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:07.011770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d91426-d164-4de6-94e6-e81f6ee9e76e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:07.012058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d91426-d164-4de6-94e6-e81f6ee9e76e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:07.012343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d91426-d164-4de6-94e6-e81f6ee9e76e None admin] PATH_INFO: `/v3/users/ac00373fb8c040dfbb033f0bab0274f2` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:07.013586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d91426-d164-4de6-94e6-e81f6ee9e76e None admin] RBAC: Authorizing `identity:get_user(user_id=ac00373fb8c040dfbb033f0bab0274f2)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:07.015025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d91426-d164-4de6-94e6-e81f6ee9e76e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:07.021154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 544/1080] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:06 2022] GET /identity/v3/users/ac00373fb8c040dfbb033f0bab0274f2 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:07.022148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d91426-d164-4de6-94e6-e81f6ee9e76e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:07.123372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ca67-3857-46bc-b7f3-1930f2994567 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:07.126968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d91426-d164-4de6-94e6-e81f6ee9e76e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:07.136859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:07.155235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:07.163869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b0a3b-f1bc-44ab-929b-8c2bd62cde0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:07.164129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97024b-d4ca-4f09-847f-4aee5bdc2c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:07.164765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b0a3b-f1bc-44ab-929b-8c2bd62cde0e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:07.164932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97024b-d4ca-4f09-847f-4aee5bdc2c88 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:07.165104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b0a3b-f1bc-44ab-929b-8c2bd62cde0e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:07.165274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97024b-d4ca-4f09-847f-4aee5bdc2c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:07.165388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b0a3b-f1bc-44ab-929b-8c2bd62cde0e None admin] PATH_INFO: `/v3/projects/a7d9e58343ff486abea5e6eb7375de83` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:07.165549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97024b-d4ca-4f09-847f-4aee5bdc2c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:07.166579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b0a3b-f1bc-44ab-929b-8c2bd62cde0e None admin] RBAC: Authorizing `identity:get_project(project_id=a7d9e58343ff486abea5e6eb7375de83)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:07.168006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b0a3b-f1bc-44ab-929b-8c2bd62cde0e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:07.175180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 545/1081] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:07 2022] GET /identity/v3/projects/a7d9e58343ff486abea5e6eb7375de83 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:07.175525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b0a3b-f1bc-44ab-929b-8c2bd62cde0e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:07.178846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97024b-d4ca-4f09-847f-4aee5bdc2c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:07.180893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97024b-d4ca-4f09-847f-4aee5bdc2c88 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-9f97024b-d4ca-4f09-847f-4aee5bdc2c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:51:07.199237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:07.201077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:07.202307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:51:07.202307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:51:07.202307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:07.202307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:07.202307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 537/1082] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:51:07.206276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97024b-d4ca-4f09-847f-4aee5bdc2c88 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:07.284428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b0a3b-f1bc-44ab-929b-8c2bd62cde0e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:07.313854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97024b-d4ca-4f09-847f-4aee5bdc2c88 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:07.327045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee6708-1804-4520-99d8-2962d35f5834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:07.328311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee6708-1804-4520-99d8-2962d35f5834 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:07.329079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee6708-1804-4520-99d8-2962d35f5834 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:07.329079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee6708-1804-4520-99d8-2962d35f5834 None admin] PATH_INFO: `/v3/projects/a7d9e58343ff486abea5e6eb7375de83/users/ac00373fb8c040dfbb033f0bab0274f2/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:07.331259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee6708-1804-4520-99d8-2962d35f5834 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7d9e58343ff486abea5e6eb7375de83, user_id=ac00373fb8c040dfbb033f0bab0274f2, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:07.332681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee6708-1804-4520-99d8-2962d35f5834 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:07.349212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 546/1083] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:51:07 2022] PUT /identity/v3/projects/a7d9e58343ff486abea5e6eb7375de83/users/ac00373fb8c040dfbb033f0bab0274f2/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:51:07.364662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54c1ce-fe94-4bb3-89e9-74680b534903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:07.365500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54c1ce-fe94-4bb3-89e9-74680b534903 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:07.365853 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54c1ce-fe94-4bb3-89e9-74680b534903 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:07.366224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54c1ce-fe94-4bb3-89e9-74680b534903 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:07.367054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54c1ce-fe94-4bb3-89e9-74680b534903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:07.367461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54c1ce-fe94-4bb3-89e9-74680b534903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:07.368967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54c1ce-fe94-4bb3-89e9-74680b534903 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:07.389213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 538/1084] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:08.470369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d07d5-9242-45ba-9fb3-480d3d281548 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:08.470901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d07d5-9242-45ba-9fb3-480d3d281548 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:08.470989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d07d5-9242-45ba-9fb3-480d3d281548 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:08.472300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 547/1085] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:08.478075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0de1e-69a0-4b62-a447-bfe575de3c50 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:08.478509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0de1e-69a0-4b62-a447-bfe575de3c50 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:08.478635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0de1e-69a0-4b62-a447-bfe575de3c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:08.496632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-45b0de1e-69a0-4b62-a447-bfe575de3c50 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:08.497844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-45b0de1e-69a0-4b62-a447-bfe575de3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:08.532315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:08.567116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 539/1086] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:08.591158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d07d5-9242-45ba-9fb3-480d3d281548 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:08.632254 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:08.702433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d07d5-9242-45ba-9fb3-480d3d281548 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:08.735727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7de1f-1285-42e9-b1f8-2cd88782f375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:08.736530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7de1f-1285-42e9-b1f8-2cd88782f375 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:08.736815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7de1f-1285-42e9-b1f8-2cd88782f375 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:08.737112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7de1f-1285-42e9-b1f8-2cd88782f375 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:08.747013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7de1f-1285-42e9-b1f8-2cd88782f375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-95d7de1f-1285-42e9-b1f8-2cd88782f375 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:08.755082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:08.756197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:08.757075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 548/1087] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jun 21 18:51:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:08.762742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0de1e-69a0-4b62-a447-bfe575de3c50 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:08.866662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0de1e-69a0-4b62-a447-bfe575de3c50 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:08.902410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1f9ff-97c5-4ad0-a1c8-9c95cdda79cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:08.903074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1f9ff-97c5-4ad0-a1c8-9c95cdda79cd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:08.903359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1f9ff-97c5-4ad0-a1c8-9c95cdda79cd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:08.903638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1f9ff-97c5-4ad0-a1c8-9c95cdda79cd None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:08.904391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1f9ff-97c5-4ad0-a1c8-9c95cdda79cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:08.904705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1f9ff-97c5-4ad0-a1c8-9c95cdda79cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:08.906107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1f9ff-97c5-4ad0-a1c8-9c95cdda79cd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:08.914743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 540/1088] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jun 21 18:51:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:08.920340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7de1f-1285-42e9-b1f8-2cd88782f375 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.013937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:09.020271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7de1f-1285-42e9-b1f8-2cd88782f375 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:09.053688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73160080-e67b-4816-be1f-6a2429457c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:09.054481 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73160080-e67b-4816-be1f-6a2429457c34 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.054794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73160080-e67b-4816-be1f-6a2429457c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.055076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73160080-e67b-4816-be1f-6a2429457c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.066650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73160080-e67b-4816-be1f-6a2429457c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:09.068096 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73160080-e67b-4816-be1f-6a2429457c34 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-73160080-e67b-4816-be1f-6a2429457c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:51:09.085547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:09.087086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:09.087996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:51:09.087996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:51:09.087996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:09.087996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:09.087996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 549/1089] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:09.093997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1f9ff-97c5-4ad0-a1c8-9c95cdda79cd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.128286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:09.201352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1f9ff-97c5-4ad0-a1c8-9c95cdda79cd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:09.235185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff83854-481d-471a-a4f3-8726651ee152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:09.235845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff83854-481d-471a-a4f3-8726651ee152 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.236133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff83854-481d-471a-a4f3-8726651ee152 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.236412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff83854-481d-471a-a4f3-8726651ee152 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.237165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff83854-481d-471a-a4f3-8726651ee152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:09.237480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff83854-481d-471a-a4f3-8726651ee152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:09.238859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff83854-481d-471a-a4f3-8726651ee152 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:09.250891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 541/1090] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:09.256605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73160080-e67b-4816-be1f-6a2429457c34 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.358424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73160080-e67b-4816-be1f-6a2429457c34 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:09.381621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553a7b1-10ed-4abc-a432-8fedb5eed0c2 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.381836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553a7b1-10ed-4abc-a432-8fedb5eed0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.382116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553a7b1-10ed-4abc-a432-8fedb5eed0c2 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.384109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 542/1091] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:51:09.389029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1c90d-82bd-45d0-b7af-4aa8273ea2a0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.389466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1c90d-82bd-45d0-b7af-4aa8273ea2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.389749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1c90d-82bd-45d0-b7af-4aa8273ea2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.394641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbaddf-f6f4-4fde-a6d2-c43a44406cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:09.395430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbaddf-f6f4-4fde-a6d2-c43a44406cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.395718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbaddf-f6f4-4fde-a6d2-c43a44406cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.395995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbaddf-f6f4-4fde-a6d2-c43a44406cb4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.403566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbaddf-f6f4-4fde-a6d2-c43a44406cb4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:09.405005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbaddf-f6f4-4fde-a6d2-c43a44406cb4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:09.405781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-22a1c90d-82bd-45d0-b7af-4aa8273ea2a0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:09.406957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-22a1c90d-82bd-45d0-b7af-4aa8273ea2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-babbaddf-f6f4-4fde-a6d2-c43a44406cb4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:51:09.418862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:09.420031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 550/1092] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jun 21 18:51:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:09.426149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbaddf-f6f4-4fde-a6d2-c43a44406cb4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.437827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:09.479480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 543/1093] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:51:09.503154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1c90d-82bd-45d0-b7af-4aa8273ea2a0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.526733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:09.526986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbaddf-f6f4-4fde-a6d2-c43a44406cb4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:09.561249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f80b88-d5ae-4c12-9d45-fd2af287b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:09.562168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f80b88-d5ae-4c12-9d45-fd2af287b08f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.562454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f80b88-d5ae-4c12-9d45-fd2af287b08f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.562725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f80b88-d5ae-4c12-9d45-fd2af287b08f None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.563557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f80b88-d5ae-4c12-9d45-fd2af287b08f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:09.563865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f80b88-d5ae-4c12-9d45-fd2af287b08f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:09.565314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f80b88-d5ae-4c12-9d45-fd2af287b08f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:09.582671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 551/1094] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jun 21 18:51:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:09.588283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f80b88-d5ae-4c12-9d45-fd2af287b08f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.603003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1c90d-82bd-45d0-b7af-4aa8273ea2a0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:09.641662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4b6de-4f7b-48b1-b5ac-dc984070837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:09.642378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4b6de-4f7b-48b1-b5ac-dc984070837c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.642661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4b6de-4f7b-48b1-b5ac-dc984070837c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.642937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4b6de-4f7b-48b1-b5ac-dc984070837c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.653338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4b6de-4f7b-48b1-b5ac-dc984070837c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:09.656237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a9a4b6de-4f7b-48b1-b5ac-dc984070837c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:09.660862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:09.662076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:09.662938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 544/1095] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:51:09.668539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4b6de-4f7b-48b1-b5ac-dc984070837c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.699315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f80b88-d5ae-4c12-9d45-fd2af287b08f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:09.734236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e2673-f07b-4da3-898f-960cc08499a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:09.734885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e2673-f07b-4da3-898f-960cc08499a4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.735172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e2673-f07b-4da3-898f-960cc08499a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.735457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e2673-f07b-4da3-898f-960cc08499a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.754054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:09.773609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4b6de-4f7b-48b1-b5ac-dc984070837c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:09.802878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e2673-f07b-4da3-898f-960cc08499a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:09.804336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e2673-f07b-4da3-898f-960cc08499a4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:09.808438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc75b71-f112-427c-84e1-d07f2e2fe0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:09.809114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc75b71-f112-427c-84e1-d07f2e2fe0ad None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.809406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc75b71-f112-427c-84e1-d07f2e2fe0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.809687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc75b71-f112-427c-84e1-d07f2e2fe0ad None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.810435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc75b71-f112-427c-84e1-d07f2e2fe0ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:09.810747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc75b71-f112-427c-84e1-d07f2e2fe0ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:09.812359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc75b71-f112-427c-84e1-d07f2e2fe0ad None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:09.820504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 545/1096] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:09.826351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc75b71-f112-427c-84e1-d07f2e2fe0ad None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.872395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:09.911711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 552/1097] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:51:09.912640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e2673-f07b-4da3-898f-960cc08499a4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.925713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc75b71-f112-427c-84e1-d07f2e2fe0ad None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:09.959620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c96ba-e97b-4b33-8cd8-fcc86293734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:09.960482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c96ba-e97b-4b33-8cd8-fcc86293734f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:09.960769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c96ba-e97b-4b33-8cd8-fcc86293734f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:09.961050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c96ba-e97b-4b33-8cd8-fcc86293734f None admin] PATH_INFO: `/v3/users/ac00373fb8c040dfbb033f0bab0274f2` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:09.962295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c96ba-e97b-4b33-8cd8-fcc86293734f None admin] RBAC: Authorizing `identity:get_user(user_id=ac00373fb8c040dfbb033f0bab0274f2)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:09.963735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c96ba-e97b-4b33-8cd8-fcc86293734f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:09.965410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 546/1098] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:09 2022] GET /identity/v3/users/ac00373fb8c040dfbb033f0bab0274f2 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:09.971242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c96ba-e97b-4b33-8cd8-fcc86293734f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:09.987200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:10.015029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e2673-f07b-4da3-898f-960cc08499a4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:10.049233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69b3d4-75dc-4f64-b181-d16619d797da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:10.050131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69b3d4-75dc-4f64-b181-d16619d797da None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:10.050481 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69b3d4-75dc-4f64-b181-d16619d797da None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:10.050822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69b3d4-75dc-4f64-b181-d16619d797da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:10.062704 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69b3d4-75dc-4f64-b181-d16619d797da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:10.063818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69b3d4-75dc-4f64-b181-d16619d797da None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:10.074166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c96ba-e97b-4b33-8cd8-fcc86293734f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-bc69b3d4-75dc-4f64-b181-d16619d797da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:51:10.081039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:10.082078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:10.083113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:51:10.083113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:51:10.083113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:51:10.083113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:10.083113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 553/1099] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:51:10.088322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69b3d4-75dc-4f64-b181-d16619d797da None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:10.107598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:10.110932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273f6fb-cbd2-4871-8d49-0fecf15eb25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:10.112039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273f6fb-cbd2-4871-8d49-0fecf15eb25d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:10.112602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273f6fb-cbd2-4871-8d49-0fecf15eb25d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:10.113073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273f6fb-cbd2-4871-8d49-0fecf15eb25d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:10.188424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273f6fb-cbd2-4871-8d49-0fecf15eb25d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:10.189232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69b3d4-75dc-4f64-b181-d16619d797da None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:10.189933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273f6fb-cbd2-4871-8d49-0fecf15eb25d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:10.203861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:10.224080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5cf7b-a8d3-4a7c-ae58-7c5340315b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:10.224753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5cf7b-a8d3-4a7c-ae58-7c5340315b46 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:10.225036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5cf7b-a8d3-4a7c-ae58-7c5340315b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:10.225348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5cf7b-a8d3-4a7c-ae58-7c5340315b46 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:10.226071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5cf7b-a8d3-4a7c-ae58-7c5340315b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:10.226382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5cf7b-a8d3-4a7c-ae58-7c5340315b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:10.228321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5cf7b-a8d3-4a7c-ae58-7c5340315b46 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:10.239737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 554/1100] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:10.245259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5cf7b-a8d3-4a7c-ae58-7c5340315b46 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:10.291531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 547/1101] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 18:51:10.297389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273f6fb-cbd2-4871-8d49-0fecf15eb25d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:10.325910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:10.350260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5cf7b-a8d3-4a7c-ae58-7c5340315b46 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:10.385649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f375-7920-405a-ae7e-b1f67f060caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:10.386554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f375-7920-405a-ae7e-b1f67f060caf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:10.386912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f375-7920-405a-ae7e-b1f67f060caf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:10.387248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f375-7920-405a-ae7e-b1f67f060caf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:10.395415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58f375-7920-405a-ae7e-b1f67f060caf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:10.397239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58f375-7920-405a-ae7e-b1f67f060caf None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:10.401790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273f6fb-cbd2-4871-8d49-0fecf15eb25d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-8c58f375-7920-405a-ae7e-b1f67f060caf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:51:10.415197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:10.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:10.417506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:51:10.417506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:51:10.417506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:51:10.417506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:51:10.417506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:51:10.417506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:51:10.417506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:51:10.417506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:10.418102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:10.418177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 555/1102] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:51:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:51:10.422990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f375-7920-405a-ae7e-b1f67f060caf None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:10.439879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4505d-8a95-4a71-880b-a0cf6a0c58d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:10.440641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4505d-8a95-4a71-880b-a0cf6a0c58d3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:10.440929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4505d-8a95-4a71-880b-a0cf6a0c58d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:10.441244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4505d-8a95-4a71-880b-a0cf6a0c58d3 None admin] PATH_INFO: `/v3/projects/a7d9e58343ff486abea5e6eb7375de83` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:10.442428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4505d-8a95-4a71-880b-a0cf6a0c58d3 None admin] RBAC: Authorizing `identity:get_project(project_id=a7d9e58343ff486abea5e6eb7375de83)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:10.443901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4505d-8a95-4a71-880b-a0cf6a0c58d3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:10.446199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 548/1103] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:10 2022] GET /identity/v3/projects/a7d9e58343ff486abea5e6eb7375de83 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:10.452847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4505d-8a95-4a71-880b-a0cf6a0c58d3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:10.522460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f375-7920-405a-ae7e-b1f67f060caf None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:10.555826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4505d-8a95-4a71-880b-a0cf6a0c58d3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:10.559018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d546a-144a-4634-bead-5e83ec6c9638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:10.559769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d546a-144a-4634-bead-5e83ec6c9638 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:10.560052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d546a-144a-4634-bead-5e83ec6c9638 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:10.560330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d546a-144a-4634-bead-5e83ec6c9638 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:10.561126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d546a-144a-4634-bead-5e83ec6c9638 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:10.561441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d546a-144a-4634-bead-5e83ec6c9638 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:10.563130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d546a-144a-4634-bead-5e83ec6c9638 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:10.570789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:10.575828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 556/1104] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:10.581719 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d546a-144a-4634-bead-5e83ec6c9638 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:10.592448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f66dd2-5e4a-4b67-91b8-ab7b52ffabfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:10.593505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f66dd2-5e4a-4b67-91b8-ab7b52ffabfe None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:10.593796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f66dd2-5e4a-4b67-91b8-ab7b52ffabfe None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:10.594077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f66dd2-5e4a-4b67-91b8-ab7b52ffabfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:10.595237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f66dd2-5e4a-4b67-91b8-ab7b52ffabfe None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=a7d9e58343ff486abea5e6eb7375de83, user.id=ac00373fb8c040dfbb033f0bab0274f2) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:10.595556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f66dd2-5e4a-4b67-91b8-ab7b52ffabfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:10.597003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f66dd2-5e4a-4b67-91b8-ab7b52ffabfe None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:10.603129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 549/1105] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:10 2022] GET /identity/v3/role_assignments?user.id=ac00373fb8c040dfbb033f0bab0274f2&scope.project.id=a7d9e58343ff486abea5e6eb7375de83&role.id=e089a0d6040040c4bcf143a680488e3d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:10.689480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d546a-144a-4634-bead-5e83ec6c9638 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:10.726320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcc5b9-419e-4c39-bd45-3f90ce24e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:10.726940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcc5b9-419e-4c39-bd45-3f90ce24e3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:10.727230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcc5b9-419e-4c39-bd45-3f90ce24e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:10.727515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcc5b9-419e-4c39-bd45-3f90ce24e3b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:10.728357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcc5b9-419e-4c39-bd45-3f90ce24e3b2 None admin] RBAC: Adding query filter params (role.id=c87ba8b8c46b4328bde6b43b256e79e3, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=dcbcec65220c4ebb95f974925fa73f34) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:10.728677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcc5b9-419e-4c39-bd45-3f90ce24e3b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:10.730183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcc5b9-419e-4c39-bd45-3f90ce24e3b2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:10.736722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 557/1106] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:10 2022] GET /identity/v3/role_assignments?user.id=dcbcec65220c4ebb95f974925fa73f34&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=c87ba8b8c46b4328bde6b43b256e79e3 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:12.649947 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bb249-4fbd-417f-8003-adc52ee24947 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:12.650468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bb249-4fbd-417f-8003-adc52ee24947 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:12.650468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bb249-4fbd-417f-8003-adc52ee24947 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:12.651812 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 550/1107] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:51:12.657127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0494-b949-4398-b4bd-914ce43df8d3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:12.657525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0494-b949-4398-b4bd-914ce43df8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:12.657701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0494-b949-4398-b4bd-914ce43df8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:12.675507 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-038d0494-b949-4398-b4bd-914ce43df8d3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:12.676698 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-038d0494-b949-4398-b4bd-914ce43df8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:12.732488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:12.743296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 558/1108] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:12.768722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bb249-4fbd-417f-8003-adc52ee24947 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:12.815627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:12.869287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bb249-4fbd-417f-8003-adc52ee24947 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:12.902743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d89c04-9ab0-4060-9c7d-5a43ba0859e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:12.903467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d89c04-9ab0-4060-9c7d-5a43ba0859e0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:12.903752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d89c04-9ab0-4060-9c7d-5a43ba0859e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:12.904029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d89c04-9ab0-4060-9c7d-5a43ba0859e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:12.905239 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d89c04-9ab0-4060-9c7d-5a43ba0859e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:12.906685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d89c04-9ab0-4060-9c7d-5a43ba0859e0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:12.908307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 551/1109] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:12.914074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d0494-b949-4398-b4bd-914ce43df8d3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:13.009619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d0494-b949-4398-b4bd-914ce43df8d3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:13.041900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bd0bb-1985-4470-9d2f-7ffcd20cfb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:13.042562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bd0bb-1985-4470-9d2f-7ffcd20cfb64 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:13.042844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bd0bb-1985-4470-9d2f-7ffcd20cfb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:13.043119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bd0bb-1985-4470-9d2f-7ffcd20cfb64 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:13.043829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867bd0bb-1985-4470-9d2f-7ffcd20cfb64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:13.045275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867bd0bb-1985-4470-9d2f-7ffcd20cfb64 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:13.065380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 559/1110] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:51:12 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:51:15.134724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f08977-2e2e-4a30-a8a1-5a471c0a18f3 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:15.135367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f08977-2e2e-4a30-a8a1-5a471c0a18f3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:15.135367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f08977-2e2e-4a30-a8a1-5a471c0a18f3 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:15.136657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 552/1111] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:15.141910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1435827-ea1c-4f2a-a10b-5cde9496d638 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:15.142300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1435827-ea1c-4f2a-a10b-5cde9496d638 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:15.142482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1435827-ea1c-4f2a-a10b-5cde9496d638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:15.159350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c1435827-ea1c-4f2a-a10b-5cde9496d638 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:15.160534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c1435827-ea1c-4f2a-a10b-5cde9496d638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:15.203212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:15.227420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 560/1112] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:15.251155 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f08977-2e2e-4a30-a8a1-5a471c0a18f3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:15.353438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f08977-2e2e-4a30-a8a1-5a471c0a18f3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:15.366447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:15.388567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc1327-4670-4e57-8437-4841f48803e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:15.389498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc1327-4670-4e57-8437-4841f48803e9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:15.389790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc1327-4670-4e57-8437-4841f48803e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:15.390069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc1327-4670-4e57-8437-4841f48803e9 None admin] PATH_INFO: `/v3/roles/8184510545604582b267e4d1871ae9bf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:15.391547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc1327-4670-4e57-8437-4841f48803e9 None admin] RBAC: Authorizing `identity:get_role(role_id=8184510545604582b267e4d1871ae9bf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:15.392928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc1327-4670-4e57-8437-4841f48803e9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:15.394341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 553/1113] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:15 2022] GET /identity/v3/roles/8184510545604582b267e4d1871ae9bf => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:15.401047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1435827-ea1c-4f2a-a10b-5cde9496d638 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:15.502509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1435827-ea1c-4f2a-a10b-5cde9496d638 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:15.535295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3b41d-ab93-47e4-a48e-05f5ba220833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:15.536152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3b41d-ab93-47e4-a48e-05f5ba220833 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:15.536434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3b41d-ab93-47e4-a48e-05f5ba220833 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:15.536711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3b41d-ab93-47e4-a48e-05f5ba220833 None admin] PATH_INFO: `/v3/users/47d12184f8f2441fa6162a2efd1098c3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:15.549359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3b41d-ab93-47e4-a48e-05f5ba220833 None admin] RBAC: Authorizing `identity:get_user(user_id=47d12184f8f2441fa6162a2efd1098c3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:15.550800 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3b41d-ab93-47e4-a48e-05f5ba220833 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:15.552421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 561/1114] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:15 2022] GET /identity/v3/users/47d12184f8f2441fa6162a2efd1098c3 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:15.558693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc1327-4670-4e57-8437-4841f48803e9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:15.657262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc1327-4670-4e57-8437-4841f48803e9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:15.691513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a54bc-b98d-4198-969c-bec76e7c7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:15.692145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a54bc-b98d-4198-969c-bec76e7c7731 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:15.692536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a54bc-b98d-4198-969c-bec76e7c7731 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:15.692821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a54bc-b98d-4198-969c-bec76e7c7731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:15.739335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:15.763805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a54bc-b98d-4198-969c-bec76e7c7731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:15.765283 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a54bc-b98d-4198-969c-bec76e7c7731 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:15.871871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 554/1115] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:51:15.877866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3b41d-ab93-47e4-a48e-05f5ba220833 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:15.912268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:15.978626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3b41d-ab93-47e4-a48e-05f5ba220833 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:16.013112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410809a7-1299-4cda-9d44-6e144d438042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:16.013960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410809a7-1299-4cda-9d44-6e144d438042 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:16.014248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410809a7-1299-4cda-9d44-6e144d438042 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:16.014516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410809a7-1299-4cda-9d44-6e144d438042 None admin] PATH_INFO: `/v3/projects/a7d9e58343ff486abea5e6eb7375de83` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:16.015678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410809a7-1299-4cda-9d44-6e144d438042 None admin] RBAC: Authorizing `identity:get_project(project_id=a7d9e58343ff486abea5e6eb7375de83)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:16.017131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410809a7-1299-4cda-9d44-6e144d438042 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:16.018793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 562/1116] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:15 2022] GET /identity/v3/projects/a7d9e58343ff486abea5e6eb7375de83 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:16.024895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a54bc-b98d-4198-969c-bec76e7c7731 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:16.107703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:16.127358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a54bc-b98d-4198-969c-bec76e7c7731 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:16.164974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b21ff-8cfd-4544-9638-a16a960a8d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:16.165635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b21ff-8cfd-4544-9638-a16a960a8d06 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:16.165845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b21ff-8cfd-4544-9638-a16a960a8d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:16.166118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b21ff-8cfd-4544-9638-a16a960a8d06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:16.166932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b21ff-8cfd-4544-9638-a16a960a8d06 None admin] RBAC: Adding query filter params (role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=a7d9e58343ff486abea5e6eb7375de83, user.id=47d12184f8f2441fa6162a2efd1098c3) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:16.167241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b21ff-8cfd-4544-9638-a16a960a8d06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:16.168661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b21ff-8cfd-4544-9638-a16a960a8d06 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:16.175015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 555/1117] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:16 2022] GET /identity/v3/role_assignments?user.id=47d12184f8f2441fa6162a2efd1098c3&scope.project.id=a7d9e58343ff486abea5e6eb7375de83&role.id=8184510545604582b267e4d1871ae9bf => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:18.161239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2ff9b-bb45-4b7f-9b9e-2ff783b13bed None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:18.161797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2ff9b-bb45-4b7f-9b9e-2ff783b13bed None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:18.161797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2ff9b-bb45-4b7f-9b9e-2ff783b13bed None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:18.163163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 563/1118] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:18.168968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411bfad-1973-4f30-9ce7-b85f502abf0d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:18.169423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411bfad-1973-4f30-9ce7-b85f502abf0d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:18.169556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411bfad-1973-4f30-9ce7-b85f502abf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:18.186575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6411bfad-1973-4f30-9ce7-b85f502abf0d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:18.188014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6411bfad-1973-4f30-9ce7-b85f502abf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:18.254167 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 556/1119] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:18.276638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2ff9b-bb45-4b7f-9b9e-2ff783b13bed None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:18.381212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2ff9b-bb45-4b7f-9b9e-2ff783b13bed None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:18.384144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:18.416452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04747c-702c-412d-bdf0-aafbaedbcdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:18.417387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04747c-702c-412d-bdf0-aafbaedbcdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:18.417721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04747c-702c-412d-bdf0-aafbaedbcdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:18.418046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04747c-702c-412d-bdf0-aafbaedbcdf0 None admin] PATH_INFO: `/v3/roles/8184510545604582b267e4d1871ae9bf` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:18.419789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04747c-702c-412d-bdf0-aafbaedbcdf0 None admin] RBAC: Authorizing `identity:get_role(role_id=8184510545604582b267e4d1871ae9bf)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:18.421497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04747c-702c-412d-bdf0-aafbaedbcdf0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:18.423042 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 564/1120] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:18 2022] GET /identity/v3/roles/8184510545604582b267e4d1871ae9bf => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:18.429059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411bfad-1973-4f30-9ce7-b85f502abf0d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:18.502085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:18.527572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411bfad-1973-4f30-9ce7-b85f502abf0d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:18.561260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54151f-552d-4590-bf1f-503a8e9ef101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:18.562372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54151f-552d-4590-bf1f-503a8e9ef101 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:18.562654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54151f-552d-4590-bf1f-503a8e9ef101 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:18.562930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54151f-552d-4590-bf1f-503a8e9ef101 None admin] PATH_INFO: `/v3/users/47d12184f8f2441fa6162a2efd1098c3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:18.564119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54151f-552d-4590-bf1f-503a8e9ef101 None admin] RBAC: Authorizing `identity:get_user(user_id=47d12184f8f2441fa6162a2efd1098c3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:18.565594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54151f-552d-4590-bf1f-503a8e9ef101 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:18.567232 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 557/1121] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:18 2022] GET /identity/v3/users/47d12184f8f2441fa6162a2efd1098c3 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:18.573561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04747c-702c-412d-bdf0-aafbaedbcdf0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:18.669043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04747c-702c-412d-bdf0-aafbaedbcdf0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:18.702185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c8e67-eb5c-4487-9387-af8375802f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:18.703029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c8e67-eb5c-4487-9387-af8375802f21 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:18.703314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c8e67-eb5c-4487-9387-af8375802f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:18.703595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c8e67-eb5c-4487-9387-af8375802f21 None admin] PATH_INFO: `/v3/projects/a7d9e58343ff486abea5e6eb7375de83` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:18.704769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c8e67-eb5c-4487-9387-af8375802f21 None admin] RBAC: Authorizing `identity:get_project(project_id=a7d9e58343ff486abea5e6eb7375de83)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:18.706232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c8e67-eb5c-4487-9387-af8375802f21 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:18.707918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 565/1122] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:18 2022] GET /identity/v3/projects/a7d9e58343ff486abea5e6eb7375de83 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:18.713950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54151f-552d-4590-bf1f-503a8e9ef101 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:18.810997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54151f-552d-4590-bf1f-503a8e9ef101 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:18.843799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808e560-2de5-40a6-9ac0-6c81c0a9dd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:18.844673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e560-2de5-40a6-9ac0-6c81c0a9dd67 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:18.844957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e560-2de5-40a6-9ac0-6c81c0a9dd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:18.845261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e560-2de5-40a6-9ac0-6c81c0a9dd67 None admin] PATH_INFO: `/v3/projects/a7d9e58343ff486abea5e6eb7375de83/users/47d12184f8f2441fa6162a2efd1098c3/roles/8184510545604582b267e4d1871ae9bf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:18.847466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808e560-2de5-40a6-9ac0-6c81c0a9dd67 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7d9e58343ff486abea5e6eb7375de83, user_id=47d12184f8f2441fa6162a2efd1098c3, role_id=8184510545604582b267e4d1871ae9bf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:18.848903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808e560-2de5-40a6-9ac0-6c81c0a9dd67 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:18.862240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 558/1123] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:51:18 2022] PUT /identity/v3/projects/a7d9e58343ff486abea5e6eb7375de83/users/47d12184f8f2441fa6162a2efd1098c3/roles/8184510545604582b267e4d1871ae9bf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:51:20.854443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26707e-8cd0-4e19-88a7-d661e1a3276c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:20.854991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26707e-8cd0-4e19-88a7-d661e1a3276c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:20.854991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26707e-8cd0-4e19-88a7-d661e1a3276c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:20.856346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 566/1124] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:20.861626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67746acd-761b-4a9a-9080-770b5715e40d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:20.862047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67746acd-761b-4a9a-9080-770b5715e40d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:20.862202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67746acd-761b-4a9a-9080-770b5715e40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:20.867763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:20.880690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-67746acd-761b-4a9a-9080-770b5715e40d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:20.881927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-67746acd-761b-4a9a-9080-770b5715e40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:20.948086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 559/1125] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:20.971609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26707e-8cd0-4e19-88a7-d661e1a3276c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:21.015345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:21.075673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26707e-8cd0-4e19-88a7-d661e1a3276c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:21.109062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f8f3-530e-4c39-8ea9-b17d27e82729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:21.109868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f8f3-530e-4c39-8ea9-b17d27e82729 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:21.110154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f8f3-530e-4c39-8ea9-b17d27e82729 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:21.110433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f8f3-530e-4c39-8ea9-b17d27e82729 None admin] PATH_INFO: `/v3/roles/8184510545604582b267e4d1871ae9bf` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:21.111917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3f8f3-530e-4c39-8ea9-b17d27e82729 None admin] RBAC: Authorizing `identity:get_role(role_id=8184510545604582b267e4d1871ae9bf)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:21.113314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3f8f3-530e-4c39-8ea9-b17d27e82729 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:21.114680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 567/1126] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:20 2022] GET /identity/v3/roles/8184510545604582b267e4d1871ae9bf => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:21.121020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67746acd-761b-4a9a-9080-770b5715e40d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:21.216239 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67746acd-761b-4a9a-9080-770b5715e40d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:21.245310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:21.250518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042bb6b-1d11-4f0d-af01-2b127646b775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:21.251384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042bb6b-1d11-4f0d-af01-2b127646b775 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:21.251672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042bb6b-1d11-4f0d-af01-2b127646b775 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:21.251953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042bb6b-1d11-4f0d-af01-2b127646b775 None admin] PATH_INFO: `/v3/users/47d12184f8f2441fa6162a2efd1098c3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:21.253194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042bb6b-1d11-4f0d-af01-2b127646b775 None admin] RBAC: Authorizing `identity:get_user(user_id=47d12184f8f2441fa6162a2efd1098c3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:21.254639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042bb6b-1d11-4f0d-af01-2b127646b775 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:21.256281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 560/1127] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:21 2022] GET /identity/v3/users/47d12184f8f2441fa6162a2efd1098c3 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:21.262147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f8f3-530e-4c39-8ea9-b17d27e82729 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:21.361625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f8f3-530e-4c39-8ea9-b17d27e82729 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:21.370476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:21.394042 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5520cd1-a069-496b-a082-32b09445965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:21.394678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5520cd1-a069-496b-a082-32b09445965d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:21.394960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5520cd1-a069-496b-a082-32b09445965d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:21.395235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5520cd1-a069-496b-a082-32b09445965d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:21.460948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5520cd1-a069-496b-a082-32b09445965d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:21.462431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5520cd1-a069-496b-a082-32b09445965d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:21.556799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 568/1128] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:51:21.562981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042bb6b-1d11-4f0d-af01-2b127646b775 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:21.662131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042bb6b-1d11-4f0d-af01-2b127646b775 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:21.696064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eff98d-94c7-4236-bc69-fec2232822c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:21.696915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eff98d-94c7-4236-bc69-fec2232822c3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:21.697231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eff98d-94c7-4236-bc69-fec2232822c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:21.697513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eff98d-94c7-4236-bc69-fec2232822c3 None admin] PATH_INFO: `/v3/projects/a7d9e58343ff486abea5e6eb7375de83` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:21.698695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eff98d-94c7-4236-bc69-fec2232822c3 None admin] RBAC: Authorizing `identity:get_project(project_id=a7d9e58343ff486abea5e6eb7375de83)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:21.700124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eff98d-94c7-4236-bc69-fec2232822c3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:21.701864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 561/1129] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:21 2022] GET /identity/v3/projects/a7d9e58343ff486abea5e6eb7375de83 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:21.707972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5520cd1-a069-496b-a082-32b09445965d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:21.718588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:21.805681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5520cd1-a069-496b-a082-32b09445965d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:21.839523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1033f50e-2f7e-44a3-b926-cf3894ca79bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:21.840170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033f50e-2f7e-44a3-b926-cf3894ca79bc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:21.840459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033f50e-2f7e-44a3-b926-cf3894ca79bc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:21.840740 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033f50e-2f7e-44a3-b926-cf3894ca79bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:21.841589 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1033f50e-2f7e-44a3-b926-cf3894ca79bc None admin] RBAC: Adding query filter params (role.id=8184510545604582b267e4d1871ae9bf, scope.project.id=a7d9e58343ff486abea5e6eb7375de83, user.id=47d12184f8f2441fa6162a2efd1098c3) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:21.841908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1033f50e-2f7e-44a3-b926-cf3894ca79bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:21.843317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1033f50e-2f7e-44a3-b926-cf3894ca79bc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:21.849004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 569/1130] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:21 2022] GET /identity/v3/role_assignments?user.id=47d12184f8f2441fa6162a2efd1098c3&scope.project.id=a7d9e58343ff486abea5e6eb7375de83&role.id=8184510545604582b267e4d1871ae9bf => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:23.906058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553d024-b5d5-43b6-8241-616c6e41aa90 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:23.906621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553d024-b5d5-43b6-8241-616c6e41aa90 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:23.906621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553d024-b5d5-43b6-8241-616c6e41aa90 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:23.907989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 562/1131] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:23.913220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7899-da6a-45ab-b4aa-4df681eb7619 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:23.914150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7899-da6a-45ab-b4aa-4df681eb7619 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:23.914336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7899-da6a-45ab-b4aa-4df681eb7619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:23.933258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-02ec7899-da6a-45ab-b4aa-4df681eb7619 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:23.934500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-02ec7899-da6a-45ab-b4aa-4df681eb7619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:24.002893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 570/1132] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:24.026090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553d024-b5d5-43b6-8241-616c6e41aa90 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:24.098144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:24.134085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553d024-b5d5-43b6-8241-616c6e41aa90 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:24.168517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893f7c0-34d7-4508-b70f-bda113af7669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:24.169263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893f7c0-34d7-4508-b70f-bda113af7669 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:24.169554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893f7c0-34d7-4508-b70f-bda113af7669 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:24.169833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893f7c0-34d7-4508-b70f-bda113af7669 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:24.171013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893f7c0-34d7-4508-b70f-bda113af7669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:24.172467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893f7c0-34d7-4508-b70f-bda113af7669 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:24.174121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 563/1133] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:24.180577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7899-da6a-45ab-b4aa-4df681eb7619 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:24.182760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:24.282080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7899-da6a-45ab-b4aa-4df681eb7619 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:24.316661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c07c8d-9369-436b-871f-b308a4d36e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:24.317360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c07c8d-9369-436b-871f-b308a4d36e89 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:24.317651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c07c8d-9369-436b-871f-b308a4d36e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:24.317930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c07c8d-9369-436b-871f-b308a4d36e89 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:24.318651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c07c8d-9369-436b-871f-b308a4d36e89 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:24.320235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c07c8d-9369-436b-871f-b308a4d36e89 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:24.348739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 571/1134] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:51:24 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:51:26.445505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205825f6-66cc-43ca-acb0-1dc68eb17c1d None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:26.446022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205825f6-66cc-43ca-acb0-1dc68eb17c1d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:26.446138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205825f6-66cc-43ca-acb0-1dc68eb17c1d None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:26.447433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 564/1135] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:26.463463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cdb91-c752-4028-accf-368c6e24b05e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:26.463927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cdb91-c752-4028-accf-368c6e24b05e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:26.464027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cdb91-c752-4028-accf-368c6e24b05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:26.480722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bc6cdb91-c752-4028-accf-368c6e24b05e None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:26.482119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bc6cdb91-c752-4028-accf-368c6e24b05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:26.534226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:26.555017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 572/1136] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:51:26.582035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205825f6-66cc-43ca-acb0-1dc68eb17c1d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:26.702192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205825f6-66cc-43ca-acb0-1dc68eb17c1d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:26.735806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec4733-78f9-4d23-a78a-d38c84909490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:26.736688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec4733-78f9-4d23-a78a-d38c84909490 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:26.737068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec4733-78f9-4d23-a78a-d38c84909490 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:26.737385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec4733-78f9-4d23-a78a-d38c84909490 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:26.738658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec4733-78f9-4d23-a78a-d38c84909490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:26.740275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec4733-78f9-4d23-a78a-d38c84909490 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:26.742508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 565/1137] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:51:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:26.748309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cdb91-c752-4028-accf-368c6e24b05e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:26.849822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cdb91-c752-4028-accf-368c6e24b05e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:26.884348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e25f2-6859-4dbd-8a73-3880431c2e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:26.885156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e25f2-6859-4dbd-8a73-3880431c2e69 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:26.885448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e25f2-6859-4dbd-8a73-3880431c2e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:26.885725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e25f2-6859-4dbd-8a73-3880431c2e69 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:26.886449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e25f2-6859-4dbd-8a73-3880431c2e69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:26.888031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e25f2-6859-4dbd-8a73-3880431c2e69 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:26.907811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 573/1138] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:51:26 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:51:29.014967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204633b-8252-4cb9-b07a-66941d3e4038 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:29.015501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204633b-8252-4cb9-b07a-66941d3e4038 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:29.015501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204633b-8252-4cb9-b07a-66941d3e4038 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:29.016880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 566/1139] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:29.022193 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a3a3d-ad62-49e4-ad0b-bc6cee6e1765 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:29.023115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a3a3d-ad62-49e4-ad0b-bc6cee6e1765 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:29.023255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a3a3d-ad62-49e4-ad0b-bc6cee6e1765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:29.040454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3a1a3a3d-ad62-49e4-ad0b-bc6cee6e1765 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:29.041688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3a1a3a3d-ad62-49e4-ad0b-bc6cee6e1765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:29.074940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:29.108314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 574/1140] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:29.132040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204633b-8252-4cb9-b07a-66941d3e4038 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:29.135887 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:29.232126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204633b-8252-4cb9-b07a-66941d3e4038 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:29.264907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832d950-4241-476d-ab70-bb999e2dbb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:29.265770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832d950-4241-476d-ab70-bb999e2dbb8e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:29.265996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832d950-4241-476d-ab70-bb999e2dbb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:29.266269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832d950-4241-476d-ab70-bb999e2dbb8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:29.276627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832d950-4241-476d-ab70-bb999e2dbb8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a832d950-4241-476d-ab70-bb999e2dbb8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:29.284184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:29.285417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:29.286409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 567/1141] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:51:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:29.291745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a3a3d-ad62-49e4-ad0b-bc6cee6e1765 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:29.390618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a3a3d-ad62-49e4-ad0b-bc6cee6e1765 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:29.424002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0841fb-ae16-424b-9642-d9640d6357f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:29.424665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0841fb-ae16-424b-9642-d9640d6357f4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:29.424954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0841fb-ae16-424b-9642-d9640d6357f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:29.425269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0841fb-ae16-424b-9642-d9640d6357f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:29.426015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0841fb-ae16-424b-9642-d9640d6357f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:29.426327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0841fb-ae16-424b-9642-d9640d6357f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:29.427689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0841fb-ae16-424b-9642-d9640d6357f4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:29.436127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 575/1142] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:29.442350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832d950-4241-476d-ab70-bb999e2dbb8e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:29.504470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:29.541466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832d950-4241-476d-ab70-bb999e2dbb8e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:29.574967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77718d-e1bd-41c5-8db1-1047e7dc35f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:29.575828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77718d-e1bd-41c5-8db1-1047e7dc35f3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:29.576112 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77718d-e1bd-41c5-8db1-1047e7dc35f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:29.576390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77718d-e1bd-41c5-8db1-1047e7dc35f3 None admin] PATH_INFO: `/v3/users/b0caa5cc26e0474091a1be332aaa3260` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:29.589939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77718d-e1bd-41c5-8db1-1047e7dc35f3 None admin] RBAC: Authorizing `identity:get_user(user_id=b0caa5cc26e0474091a1be332aaa3260)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:29.591378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77718d-e1bd-41c5-8db1-1047e7dc35f3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:29.593043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 568/1143] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:29 2022] GET /identity/v3/users/b0caa5cc26e0474091a1be332aaa3260 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:29.599046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0841fb-ae16-424b-9642-d9640d6357f4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:29.632434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:29.701036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0841fb-ae16-424b-9642-d9640d6357f4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:29.734248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f057a-945a-4a3b-a2fb-45ad55a9fcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:29.734896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f057a-945a-4a3b-a2fb-45ad55a9fcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:29.735183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f057a-945a-4a3b-a2fb-45ad55a9fcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:29.735465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f057a-945a-4a3b-a2fb-45ad55a9fcb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:29.801632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f057a-945a-4a3b-a2fb-45ad55a9fcb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:29.802972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f057a-945a-4a3b-a2fb-45ad55a9fcb1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:29.833557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:29.902269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 576/1144] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:51:29.908200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77718d-e1bd-41c5-8db1-1047e7dc35f3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:30.010569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77718d-e1bd-41c5-8db1-1047e7dc35f3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:30.033222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:30.045400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb2f2b-3fcb-4486-a1c4-56d196182fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:30.046263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb2f2b-3fcb-4486-a1c4-56d196182fdf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:30.046551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb2f2b-3fcb-4486-a1c4-56d196182fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:30.046833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb2f2b-3fcb-4486-a1c4-56d196182fdf None admin] PATH_INFO: `/v3/projects/66fa907ffbee45ffa057b2b246aa6b25` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:30.048015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb2f2b-3fcb-4486-a1c4-56d196182fdf None admin] RBAC: Authorizing `identity:get_project(project_id=66fa907ffbee45ffa057b2b246aa6b25)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:30.049487 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb2f2b-3fcb-4486-a1c4-56d196182fdf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:30.051211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 569/1145] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:29 2022] GET /identity/v3/projects/66fa907ffbee45ffa057b2b246aa6b25 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:30.057980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f057a-945a-4a3b-a2fb-45ad55a9fcb1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:30.156386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f057a-945a-4a3b-a2fb-45ad55a9fcb1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:30.186181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:30.192154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a2fad-5000-4694-aabd-93f060e99b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:30.192805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a2fad-5000-4694-aabd-93f060e99b06 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:30.193100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a2fad-5000-4694-aabd-93f060e99b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:30.193400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a2fad-5000-4694-aabd-93f060e99b06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:30.194226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a2fad-5000-4694-aabd-93f060e99b06 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=66fa907ffbee45ffa057b2b246aa6b25, user.id=b0caa5cc26e0474091a1be332aaa3260) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:30.195055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a2fad-5000-4694-aabd-93f060e99b06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:30.196471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a2fad-5000-4694-aabd-93f060e99b06 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:30.202321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 577/1146] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:30 2022] GET /identity/v3/role_assignments?user.id=b0caa5cc26e0474091a1be332aaa3260&scope.project.id=66fa907ffbee45ffa057b2b246aa6b25&role.id=e089a0d6040040c4bcf143a680488e3d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:32.238785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d18b5-4ef5-4515-a887-aa42ef416cfe None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:32.239340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d18b5-4ef5-4515-a887-aa42ef416cfe None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:32.239340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d18b5-4ef5-4515-a887-aa42ef416cfe None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:32.240763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 570/1147] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:32.247649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff03f-bde6-4efb-8405-8b9096f95cf0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:32.248177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff03f-bde6-4efb-8405-8b9096f95cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:32.248692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff03f-bde6-4efb-8405-8b9096f95cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:32.267087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8a9ff03f-bde6-4efb-8405-8b9096f95cf0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:32.268531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8a9ff03f-bde6-4efb-8405-8b9096f95cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:32.339296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 578/1148] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:32.362336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d18b5-4ef5-4515-a887-aa42ef416cfe None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:32.474130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d18b5-4ef5-4515-a887-aa42ef416cfe None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:32.509747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c0b57-5eec-4c08-8657-c7cad906a256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:32.510451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c0b57-5eec-4c08-8657-c7cad906a256 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:32.510753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c0b57-5eec-4c08-8657-c7cad906a256 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:32.511011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c0b57-5eec-4c08-8657-c7cad906a256 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:32.521095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c0b57-5eec-4c08-8657-c7cad906a256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-069c0b57-5eec-4c08-8657-c7cad906a256 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:32.529571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:32.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:32.531700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 571/1149] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:51:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:32.537420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ff03f-bde6-4efb-8405-8b9096f95cf0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:32.641352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ff03f-bde6-4efb-8405-8b9096f95cf0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:32.668515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:32.677773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d64315-044a-4c78-8c8c-702ba1ebba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:32.678440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d64315-044a-4c78-8c8c-702ba1ebba5c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:32.678730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d64315-044a-4c78-8c8c-702ba1ebba5c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:32.679009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d64315-044a-4c78-8c8c-702ba1ebba5c None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:32.679760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d64315-044a-4c78-8c8c-702ba1ebba5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:32.680072 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d64315-044a-4c78-8c8c-702ba1ebba5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:32.681482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d64315-044a-4c78-8c8c-702ba1ebba5c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:32.689939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 579/1150] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:51:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:32.695801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c0b57-5eec-4c08-8657-c7cad906a256 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:32.746124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:32.797396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c0b57-5eec-4c08-8657-c7cad906a256 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:32.833325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c626a7-1802-4d03-a950-04fa95789967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:32.834190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c626a7-1802-4d03-a950-04fa95789967 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:32.834472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c626a7-1802-4d03-a950-04fa95789967 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:32.834741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c626a7-1802-4d03-a950-04fa95789967 None admin] PATH_INFO: `/v3/users/b0caa5cc26e0474091a1be332aaa3260` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:32.836410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c626a7-1802-4d03-a950-04fa95789967 None admin] RBAC: Authorizing `identity:get_user(user_id=b0caa5cc26e0474091a1be332aaa3260)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:32.837872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c626a7-1802-4d03-a950-04fa95789967 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:32.839504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 572/1151] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:32 2022] GET /identity/v3/users/b0caa5cc26e0474091a1be332aaa3260 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:32.845491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d64315-044a-4c78-8c8c-702ba1ebba5c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:32.947220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d64315-044a-4c78-8c8c-702ba1ebba5c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:32.979363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeff24-ffa0-41ad-b7b9-0d2fe7fc35cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:32.980199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeff24-ffa0-41ad-b7b9-0d2fe7fc35cf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:32.980482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeff24-ffa0-41ad-b7b9-0d2fe7fc35cf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:32.980759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeff24-ffa0-41ad-b7b9-0d2fe7fc35cf None admin] PATH_INFO: `/v3/projects/66fa907ffbee45ffa057b2b246aa6b25` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:32.981954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeff24-ffa0-41ad-b7b9-0d2fe7fc35cf None admin] RBAC: Authorizing `identity:get_project(project_id=66fa907ffbee45ffa057b2b246aa6b25)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:32.983362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeff24-ffa0-41ad-b7b9-0d2fe7fc35cf None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:32.985026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 580/1152] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:32 2022] GET /identity/v3/projects/66fa907ffbee45ffa057b2b246aa6b25 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:32.990953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c626a7-1802-4d03-a950-04fa95789967 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:33.088668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c626a7-1802-4d03-a950-04fa95789967 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:33.104877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:33.126306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5fc82-8dcc-45c7-af21-6d3d9a656495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:33.127194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5fc82-8dcc-45c7-af21-6d3d9a656495 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:33.127475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5fc82-8dcc-45c7-af21-6d3d9a656495 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:33.127748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5fc82-8dcc-45c7-af21-6d3d9a656495 None admin] PATH_INFO: `/v3/projects/66fa907ffbee45ffa057b2b246aa6b25/users/b0caa5cc26e0474091a1be332aaa3260/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:33.129746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5fc82-8dcc-45c7-af21-6d3d9a656495 None admin] RBAC: Authorizing `identity:create_grant(project_id=66fa907ffbee45ffa057b2b246aa6b25, user_id=b0caa5cc26e0474091a1be332aaa3260, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:33.131187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5fc82-8dcc-45c7-af21-6d3d9a656495 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:33.146456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 573/1153] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:51:32 2022] PUT /identity/v3/projects/66fa907ffbee45ffa057b2b246aa6b25/users/b0caa5cc26e0474091a1be332aaa3260/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:51:35.170678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bea8b-dbb0-4487-b9da-48ced91cf06c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:35.171207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bea8b-dbb0-4487-b9da-48ced91cf06c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:35.171322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bea8b-dbb0-4487-b9da-48ced91cf06c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:35.172599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 581/1154] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:35.177918 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecda5cd-babb-4af5-bff6-822535b13b58 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:35.178303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecda5cd-babb-4af5-bff6-822535b13b58 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:35.178485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecda5cd-babb-4af5-bff6-822535b13b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:35.195886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7ecda5cd-babb-4af5-bff6-822535b13b58 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:35.197072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7ecda5cd-babb-4af5-bff6-822535b13b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:35.261938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 574/1155] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:35.286167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bea8b-dbb0-4487-b9da-48ced91cf06c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:35.346085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:35.390641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bea8b-dbb0-4487-b9da-48ced91cf06c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:35.423899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97e2e3-5b40-492f-90fe-3dc2d30d5f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:35.424610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97e2e3-5b40-492f-90fe-3dc2d30d5f50 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:35.424899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97e2e3-5b40-492f-90fe-3dc2d30d5f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:35.425215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97e2e3-5b40-492f-90fe-3dc2d30d5f50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:35.435163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97e2e3-5b40-492f-90fe-3dc2d30d5f50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-ea97e2e3-5b40-492f-90fe-3dc2d30d5f50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:35.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:35.444080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:35.445547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 582/1156] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:51:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:51:35.451278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecda5cd-babb-4af5-bff6-822535b13b58 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:35.552866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecda5cd-babb-4af5-bff6-822535b13b58 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:35.562759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:35.588297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cf008-08a3-4038-9d4b-6e9473b783d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:35.588953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cf008-08a3-4038-9d4b-6e9473b783d6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:35.589267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cf008-08a3-4038-9d4b-6e9473b783d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:35.589546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cf008-08a3-4038-9d4b-6e9473b783d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:35.590286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cf008-08a3-4038-9d4b-6e9473b783d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:35.590593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cf008-08a3-4038-9d4b-6e9473b783d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:35.592060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cf008-08a3-4038-9d4b-6e9473b783d6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:35.600006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 575/1157] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:51:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:35.606134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97e2e3-5b40-492f-90fe-3dc2d30d5f50 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:35.704486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97e2e3-5b40-492f-90fe-3dc2d30d5f50 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:35.720122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:35.737953 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861aa0e-abaa-42c0-99a5-c46bd743baa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:35.738822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861aa0e-abaa-42c0-99a5-c46bd743baa5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:35.739109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861aa0e-abaa-42c0-99a5-c46bd743baa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:35.739385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861aa0e-abaa-42c0-99a5-c46bd743baa5 None admin] PATH_INFO: `/v3/users/b0caa5cc26e0474091a1be332aaa3260` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:35.740577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861aa0e-abaa-42c0-99a5-c46bd743baa5 None admin] RBAC: Authorizing `identity:get_user(user_id=b0caa5cc26e0474091a1be332aaa3260)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:35.742573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861aa0e-abaa-42c0-99a5-c46bd743baa5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:35.744205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 583/1158] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:35 2022] GET /identity/v3/users/b0caa5cc26e0474091a1be332aaa3260 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:35.750281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cf008-08a3-4038-9d4b-6e9473b783d6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:35.850016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cf008-08a3-4038-9d4b-6e9473b783d6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:35.883523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652b39c-e4da-403f-8c67-e0bc6e98011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:35.884169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652b39c-e4da-403f-8c67-e0bc6e98011e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:35.884457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652b39c-e4da-403f-8c67-e0bc6e98011e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:35.884736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652b39c-e4da-403f-8c67-e0bc6e98011e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:35.922957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:35.952091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652b39c-e4da-403f-8c67-e0bc6e98011e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:35.953601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652b39c-e4da-403f-8c67-e0bc6e98011e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:36.048863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 576/1159] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:51:36.055291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861aa0e-abaa-42c0-99a5-c46bd743baa5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:36.154904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861aa0e-abaa-42c0-99a5-c46bd743baa5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:36.189467 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcbbe4-50f9-43be-b43d-be6a6df95747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:36.190313 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcbbe4-50f9-43be-b43d-be6a6df95747 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:36.190592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcbbe4-50f9-43be-b43d-be6a6df95747 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:36.190916 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcbbe4-50f9-43be-b43d-be6a6df95747 None admin] PATH_INFO: `/v3/projects/66fa907ffbee45ffa057b2b246aa6b25` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:36.192246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcbbe4-50f9-43be-b43d-be6a6df95747 None admin] RBAC: Authorizing `identity:get_project(project_id=66fa907ffbee45ffa057b2b246aa6b25)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:36.193700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcbbe4-50f9-43be-b43d-be6a6df95747 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:36.195547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 584/1160] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:36 2022] GET /identity/v3/projects/66fa907ffbee45ffa057b2b246aa6b25 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:36.201903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652b39c-e4da-403f-8c67-e0bc6e98011e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:36.269327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:36.300827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652b39c-e4da-403f-8c67-e0bc6e98011e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:36.333933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb3df1-47ff-4621-a9fd-6869249e05f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:36.334591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb3df1-47ff-4621-a9fd-6869249e05f6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:36.335390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb3df1-47ff-4621-a9fd-6869249e05f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:36.335663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb3df1-47ff-4621-a9fd-6869249e05f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:36.336488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb3df1-47ff-4621-a9fd-6869249e05f6 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=66fa907ffbee45ffa057b2b246aa6b25, user.id=b0caa5cc26e0474091a1be332aaa3260) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:36.336804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb3df1-47ff-4621-a9fd-6869249e05f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:36.338278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb3df1-47ff-4621-a9fd-6869249e05f6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:36.344119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 577/1161] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:36 2022] GET /identity/v3/role_assignments?user.id=b0caa5cc26e0474091a1be332aaa3260&scope.project.id=66fa907ffbee45ffa057b2b246aa6b25&role.id=e089a0d6040040c4bcf143a680488e3d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:38.382526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c0916-0412-4f6b-9c93-eb79f20cca7b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:38.383079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c0916-0412-4f6b-9c93-eb79f20cca7b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:38.383192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c0916-0412-4f6b-9c93-eb79f20cca7b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:38.384746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 585/1162] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:38.390394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782030f-8525-4f0b-9536-32341b185198 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:38.390784 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782030f-8525-4f0b-9536-32341b185198 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:38.390962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782030f-8525-4f0b-9536-32341b185198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:38.407880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e782030f-8525-4f0b-9536-32341b185198 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:38.409077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e782030f-8525-4f0b-9536-32341b185198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:38.475743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 578/1163] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:38.501430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c0916-0412-4f6b-9c93-eb79f20cca7b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:38.558383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:38.607425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c0916-0412-4f6b-9c93-eb79f20cca7b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:38.656094 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad784612-75cf-482f-8027-ddd3952f44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:38.657004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad784612-75cf-482f-8027-ddd3952f44cd None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:38.657463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad784612-75cf-482f-8027-ddd3952f44cd None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:38.657912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad784612-75cf-482f-8027-ddd3952f44cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:38.745598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad784612-75cf-482f-8027-ddd3952f44cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:38.747861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad784612-75cf-482f-8027-ddd3952f44cd None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:38.817741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:38.877882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 586/1164] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:51:38.884779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782030f-8525-4f0b-9536-32341b185198 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:38.945110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:38.983575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782030f-8525-4f0b-9536-32341b185198 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:39.017037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4ca9c-3526-4848-a38a-e31566b120a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:39.017862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4ca9c-3526-4848-a38a-e31566b120a6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:39.018142 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4ca9c-3526-4848-a38a-e31566b120a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:39.018416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4ca9c-3526-4848-a38a-e31566b120a6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:39.024919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4ca9c-3526-4848-a38a-e31566b120a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:39.026521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4ca9c-3526-4848-a38a-e31566b120a6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-10b4ca9c-3526-4848-a38a-e31566b120a6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:39.040845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:39.042201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 18:51:39.043288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:39.045269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 579/1165] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:51:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:39.048314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad784612-75cf-482f-8027-ddd3952f44cd None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:39.151103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad784612-75cf-482f-8027-ddd3952f44cd None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:39.185525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efdc04-7481-4ae3-a893-c0be0203829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:39.186179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efdc04-7481-4ae3-a893-c0be0203829c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:39.186462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efdc04-7481-4ae3-a893-c0be0203829c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:39.186996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efdc04-7481-4ae3-a893-c0be0203829c None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:39.187712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efdc04-7481-4ae3-a893-c0be0203829c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:39.188020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efdc04-7481-4ae3-a893-c0be0203829c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:39.189403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efdc04-7481-4ae3-a893-c0be0203829c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:39.196745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 587/1166] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jun 21 18:51:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:41.249113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0f5be-0e58-4572-a77f-16382b2056e5 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:41.249763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0f5be-0e58-4572-a77f-16382b2056e5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:41.249763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0f5be-0e58-4572-a77f-16382b2056e5 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:41.251071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 580/1167] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:41.256965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26033f3-64cb-4df2-a348-b3f063ad769d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:41.257671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26033f3-64cb-4df2-a348-b3f063ad769d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:41.257671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26033f3-64cb-4df2-a348-b3f063ad769d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:41.275073 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d26033f3-64cb-4df2-a348-b3f063ad769d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:41.276535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d26033f3-64cb-4df2-a348-b3f063ad769d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:41.287709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:41.347224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 588/1168] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 18:51:41.370194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0f5be-0e58-4572-a77f-16382b2056e5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:41.476202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0f5be-0e58-4572-a77f-16382b2056e5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:41.499917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:41.511689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f972a-f145-47b6-b85f-26996de47f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:41.512368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f972a-f145-47b6-b85f-26996de47f29 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:41.512676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f972a-f145-47b6-b85f-26996de47f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:41.512980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f972a-f145-47b6-b85f-26996de47f29 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:41.513840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f972a-f145-47b6-b85f-26996de47f29 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:41.515215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f972a-f145-47b6-b85f-26996de47f29 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:41.530319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 581/1169] 10.222.0.70 () {60 vars in 1138 bytes} [Tue Jun 21 18:51:41 2022] POST /identity/v3/domains => generated 267 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:51:43.661760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aa32c-17bf-480f-88c1-904c82a31b3b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:43.662286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aa32c-17bf-480f-88c1-904c82a31b3b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:43.662286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aa32c-17bf-480f-88c1-904c82a31b3b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:43.664149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 589/1170] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:43.669825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f382c10-eca6-4059-b4a0-6d421addd6a2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:43.670362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f382c10-eca6-4059-b4a0-6d421addd6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:43.670471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f382c10-eca6-4059-b4a0-6d421addd6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:43.687828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9f382c10-eca6-4059-b4a0-6d421addd6a2 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:43.689029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9f382c10-eca6-4059-b4a0-6d421addd6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:43.757581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 582/1171] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:43.781654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aa32c-17bf-480f-88c1-904c82a31b3b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:43.886960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aa32c-17bf-480f-88c1-904c82a31b3b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:43.920573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62068b2f-066c-46aa-b45c-aa21243b1d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:43.921392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62068b2f-066c-46aa-b45c-aa21243b1d80 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:43.921679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62068b2f-066c-46aa-b45c-aa21243b1d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:43.921960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62068b2f-066c-46aa-b45c-aa21243b1d80 None admin] PATH_INFO: `/v3/domains/7cf356c3755f4823b6b16cd52d569a2d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:43.933103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62068b2f-066c-46aa-b45c-aa21243b1d80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7cf356c3755f4823b6b16cd52d569a2d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:43.934571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62068b2f-066c-46aa-b45c-aa21243b1d80 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:43.936321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 590/1172] 10.222.0.70 () {56 vars in 1183 bytes} [Tue Jun 21 18:51:43 2022] GET /identity/v3/domains/7cf356c3755f4823b6b16cd52d569a2d => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:43.942726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f382c10-eca6-4059-b4a0-6d421addd6a2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:44.050743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f382c10-eca6-4059-b4a0-6d421addd6a2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:44.065708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:44.088599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4993f-5bbc-42b4-91cc-76d63948d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:44.089304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4993f-5bbc-42b4-91cc-76d63948d515 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:44.089594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4993f-5bbc-42b4-91cc-76d63948d515 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:44.089871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4993f-5bbc-42b4-91cc-76d63948d515 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:44.090622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4993f-5bbc-42b4-91cc-76d63948d515 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:44.092172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4993f-5bbc-42b4-91cc-76d63948d515 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:44.119293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 583/1173] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:51:43 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:51:46.242795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d09226-daef-4486-8e22-8a0ef1cf16f4 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:46.243444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d09226-daef-4486-8e22-8a0ef1cf16f4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:46.243444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d09226-daef-4486-8e22-8a0ef1cf16f4 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:46.244712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 591/1174] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:46.250798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352753-f13a-4c18-aaed-68fd34787475 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:46.251249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352753-f13a-4c18-aaed-68fd34787475 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:46.251363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352753-f13a-4c18-aaed-68fd34787475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:46.269063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0e352753-f13a-4c18-aaed-68fd34787475 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:46.270331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0e352753-f13a-4c18-aaed-68fd34787475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:46.341113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 584/1175] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:46.366785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d09226-daef-4486-8e22-8a0ef1cf16f4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:46.376671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:46.478282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d09226-daef-4486-8e22-8a0ef1cf16f4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:46.512678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e440a7-246e-49c0-86c7-b3e344ed6dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:46.513638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e440a7-246e-49c0-86c7-b3e344ed6dbe None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:46.513951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e440a7-246e-49c0-86c7-b3e344ed6dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:46.514235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e440a7-246e-49c0-86c7-b3e344ed6dbe None admin] PATH_INFO: `/v3/domains/7cf356c3755f4823b6b16cd52d569a2d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:46.515697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e440a7-246e-49c0-86c7-b3e344ed6dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=7cf356c3755f4823b6b16cd52d569a2d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:46.517428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e440a7-246e-49c0-86c7-b3e344ed6dbe None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:46.519064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 592/1176] 10.222.0.70 () {56 vars in 1183 bytes} [Tue Jun 21 18:51:46 2022] GET /identity/v3/domains/7cf356c3755f4823b6b16cd52d569a2d => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:46.524890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e352753-f13a-4c18-aaed-68fd34787475 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:46.629614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:46.636475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e352753-f13a-4c18-aaed-68fd34787475 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:46.671673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed8f73-40db-4a7f-add4-3dbd574b2dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:46.673972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed8f73-40db-4a7f-add4-3dbd574b2dd8 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:46.673972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed8f73-40db-4a7f-add4-3dbd574b2dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:46.673972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed8f73-40db-4a7f-add4-3dbd574b2dd8 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:46.673972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed8f73-40db-4a7f-add4-3dbd574b2dd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:46.675015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed8f73-40db-4a7f-add4-3dbd574b2dd8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:46.695435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 585/1177] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:51:46 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:51:48.813734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4252c-975c-4e0f-bc6e-b347b2958601 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:48.814290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4252c-975c-4e0f-bc6e-b347b2958601 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:48.814392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4252c-975c-4e0f-bc6e-b347b2958601 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:48.815751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 593/1178] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:48.821016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79073736-aa5a-425d-8236-38d77874280f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:48.821480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79073736-aa5a-425d-8236-38d77874280f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:48.821618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79073736-aa5a-425d-8236-38d77874280f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:48.838227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-79073736-aa5a-425d-8236-38d77874280f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:48.839425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-79073736-aa5a-425d-8236-38d77874280f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:48.905960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 586/1179] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:48.931438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4252c-975c-4e0f-bc6e-b347b2958601 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:49.006246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:49.032144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4252c-975c-4e0f-bc6e-b347b2958601 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:49.066178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe2024-dcb6-4408-a6c6-1085ad794bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:49.066894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe2024-dcb6-4408-a6c6-1085ad794bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:49.067178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe2024-dcb6-4408-a6c6-1085ad794bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:49.067456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe2024-dcb6-4408-a6c6-1085ad794bd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:49.077372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe2024-dcb6-4408-a6c6-1085ad794bd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-ccbe2024-dcb6-4408-a6c6-1085ad794bd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:49.084935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:49.086735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:49.087730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 594/1180] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:51:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:49.092761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79073736-aa5a-425d-8236-38d77874280f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:49.176835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:49.196235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79073736-aa5a-425d-8236-38d77874280f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:49.232863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91452f7e-4355-48ad-b08b-bdc0ebf46d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:49.233558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91452f7e-4355-48ad-b08b-bdc0ebf46d1e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:49.233877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91452f7e-4355-48ad-b08b-bdc0ebf46d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:49.234178 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91452f7e-4355-48ad-b08b-bdc0ebf46d1e None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:49.235054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91452f7e-4355-48ad-b08b-bdc0ebf46d1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:49.235377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91452f7e-4355-48ad-b08b-bdc0ebf46d1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:49.236791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91452f7e-4355-48ad-b08b-bdc0ebf46d1e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:49.245716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 587/1181] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:51:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:49.252227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe2024-dcb6-4408-a6c6-1085ad794bd6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:49.367921 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe2024-dcb6-4408-a6c6-1085ad794bd6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:49.405884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fcb20-8102-4017-8994-b11c863ea8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:49.406748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fcb20-8102-4017-8994-b11c863ea8fc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:49.407036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fcb20-8102-4017-8994-b11c863ea8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:49.407315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fcb20-8102-4017-8994-b11c863ea8fc None admin] PATH_INFO: `/v3/users/cacbb3724dbd4ebeb1e6a7e9f4c0a67c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:49.409471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:49.423388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fcb20-8102-4017-8994-b11c863ea8fc None admin] RBAC: Authorizing `identity:get_user(user_id=cacbb3724dbd4ebeb1e6a7e9f4c0a67c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:49.424844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fcb20-8102-4017-8994-b11c863ea8fc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:49.426610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 595/1182] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:49 2022] GET /identity/v3/users/cacbb3724dbd4ebeb1e6a7e9f4c0a67c => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:51:49.432401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91452f7e-4355-48ad-b08b-bdc0ebf46d1e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:49.541163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91452f7e-4355-48ad-b08b-bdc0ebf46d1e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:49.576342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f985f6-d02a-481f-8b21-2d1b82be72f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:49.576911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f985f6-d02a-481f-8b21-2d1b82be72f4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:49.577225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f985f6-d02a-481f-8b21-2d1b82be72f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:49.577507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f985f6-d02a-481f-8b21-2d1b82be72f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:49.593444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:49.646400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f985f6-d02a-481f-8b21-2d1b82be72f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:49.647858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f985f6-d02a-481f-8b21-2d1b82be72f4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:49.743282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 588/1183] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:51:49.749249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fcb20-8102-4017-8994-b11c863ea8fc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:49.846436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fcb20-8102-4017-8994-b11c863ea8fc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:49.879585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf0171-3adf-47a2-8650-1fb16f75aaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:49.880442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf0171-3adf-47a2-8650-1fb16f75aaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:49.880725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf0171-3adf-47a2-8650-1fb16f75aaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:49.881005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf0171-3adf-47a2-8650-1fb16f75aaf9 None admin] PATH_INFO: `/v3/projects/738720f6151045b9b3b46f435e5382cf` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:49.882216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf0171-3adf-47a2-8650-1fb16f75aaf9 None admin] RBAC: Authorizing `identity:get_project(project_id=738720f6151045b9b3b46f435e5382cf)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:49.884264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf0171-3adf-47a2-8650-1fb16f75aaf9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:49.886005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 596/1184] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:49 2022] GET /identity/v3/projects/738720f6151045b9b3b46f435e5382cf => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:49.892123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f985f6-d02a-481f-8b21-2d1b82be72f4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:49.936842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:49.990527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f985f6-d02a-481f-8b21-2d1b82be72f4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:50.024601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb356088-0f6d-4b8a-9c91-df13b5c67745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:50.025273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb356088-0f6d-4b8a-9c91-df13b5c67745 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:50.025565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb356088-0f6d-4b8a-9c91-df13b5c67745 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:50.025846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb356088-0f6d-4b8a-9c91-df13b5c67745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:50.026673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb356088-0f6d-4b8a-9c91-df13b5c67745 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=738720f6151045b9b3b46f435e5382cf, user.id=cacbb3724dbd4ebeb1e6a7e9f4c0a67c) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:50.026988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb356088-0f6d-4b8a-9c91-df13b5c67745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:50.028417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb356088-0f6d-4b8a-9c91-df13b5c67745 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:50.033851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 589/1185] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:49 2022] GET /identity/v3/role_assignments?user.id=cacbb3724dbd4ebeb1e6a7e9f4c0a67c&scope.project.id=738720f6151045b9b3b46f435e5382cf&role.id=e089a0d6040040c4bcf143a680488e3d => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:52.135791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f19001-f7a7-494b-90eb-218cc96fcb85 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:52.136344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f19001-f7a7-494b-90eb-218cc96fcb85 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:52.136344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f19001-f7a7-494b-90eb-218cc96fcb85 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:52.137745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 597/1186] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:52.144192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622aacd-128b-4077-82dd-703a6ef7ad8b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:52.144618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622aacd-128b-4077-82dd-703a6ef7ad8b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:52.144767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622aacd-128b-4077-82dd-703a6ef7ad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:52.162759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b622aacd-128b-4077-82dd-703a6ef7ad8b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:52.164592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b622aacd-128b-4077-82dd-703a6ef7ad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:52.233613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 590/1187] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:52.256802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f19001-f7a7-494b-90eb-218cc96fcb85 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:52.318245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:52.368626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f19001-f7a7-494b-90eb-218cc96fcb85 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:52.411557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929d597-25a5-40c6-9696-edf3b09cea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:52.412273 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929d597-25a5-40c6-9696-edf3b09cea9d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:52.412559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929d597-25a5-40c6-9696-edf3b09cea9d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:52.412839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929d597-25a5-40c6-9696-edf3b09cea9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:52.424041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929d597-25a5-40c6-9696-edf3b09cea9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a929d597-25a5-40c6-9696-edf3b09cea9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:52.431685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:52.432839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:51:52.433928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 598/1188] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:51:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:52.439346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622aacd-128b-4077-82dd-703a6ef7ad8b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:52.480297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:52.550374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622aacd-128b-4077-82dd-703a6ef7ad8b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:52.586285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9667bc8-274b-438e-b595-2776efe53fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:52.586952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9667bc8-274b-438e-b595-2776efe53fac None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:52.587241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9667bc8-274b-438e-b595-2776efe53fac None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:52.587525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9667bc8-274b-438e-b595-2776efe53fac None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:52.588278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9667bc8-274b-438e-b595-2776efe53fac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:52.588591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9667bc8-274b-438e-b595-2776efe53fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:52.589992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9667bc8-274b-438e-b595-2776efe53fac None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:52.598160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 591/1189] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:51:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:52.603980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929d597-25a5-40c6-9696-edf3b09cea9d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:52.708863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929d597-25a5-40c6-9696-edf3b09cea9d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:52.728389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:52.742377 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361bf9b-1bec-4f67-acf8-9680edebe72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:52.743245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361bf9b-1bec-4f67-acf8-9680edebe72a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:52.743528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361bf9b-1bec-4f67-acf8-9680edebe72a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:52.743807 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361bf9b-1bec-4f67-acf8-9680edebe72a None admin] PATH_INFO: `/v3/users/cacbb3724dbd4ebeb1e6a7e9f4c0a67c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:52.744984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361bf9b-1bec-4f67-acf8-9680edebe72a None admin] RBAC: Authorizing `identity:get_user(user_id=cacbb3724dbd4ebeb1e6a7e9f4c0a67c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:52.746447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361bf9b-1bec-4f67-acf8-9680edebe72a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:52.748075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 599/1190] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:52 2022] GET /identity/v3/users/cacbb3724dbd4ebeb1e6a7e9f4c0a67c => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:52.754056 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9667bc8-274b-438e-b595-2776efe53fac None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:52.849448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:52.852555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9667bc8-274b-438e-b595-2776efe53fac None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:52.887004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cad5e-f8d6-4f7e-81f8-d0ffe18763c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:52.887852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cad5e-f8d6-4f7e-81f8-d0ffe18763c6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:52.888136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cad5e-f8d6-4f7e-81f8-d0ffe18763c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:52.888414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cad5e-f8d6-4f7e-81f8-d0ffe18763c6 None admin] PATH_INFO: `/v3/projects/738720f6151045b9b3b46f435e5382cf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:52.889609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cad5e-f8d6-4f7e-81f8-d0ffe18763c6 None admin] RBAC: Authorizing `identity:get_project(project_id=738720f6151045b9b3b46f435e5382cf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:52.891045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cad5e-f8d6-4f7e-81f8-d0ffe18763c6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:52.892747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 592/1191] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:52 2022] GET /identity/v3/projects/738720f6151045b9b3b46f435e5382cf => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:52.898747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361bf9b-1bec-4f67-acf8-9680edebe72a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:52.997731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361bf9b-1bec-4f67-acf8-9680edebe72a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:53.030918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ec82d-21a8-4841-9a5a-c799a8416197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:53.031798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ec82d-21a8-4841-9a5a-c799a8416197 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:53.032086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ec82d-21a8-4841-9a5a-c799a8416197 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:53.032365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ec82d-21a8-4841-9a5a-c799a8416197 None admin] PATH_INFO: `/v3/projects/738720f6151045b9b3b46f435e5382cf/users/cacbb3724dbd4ebeb1e6a7e9f4c0a67c/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:53.034315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ec82d-21a8-4841-9a5a-c799a8416197 None admin] RBAC: Authorizing `identity:create_grant(project_id=738720f6151045b9b3b46f435e5382cf, user_id=cacbb3724dbd4ebeb1e6a7e9f4c0a67c, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:53.035745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ec82d-21a8-4841-9a5a-c799a8416197 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:53.048676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 600/1192] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:51:52 2022] PUT /identity/v3/projects/738720f6151045b9b3b46f435e5382cf/users/cacbb3724dbd4ebeb1e6a7e9f4c0a67c/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:51:55.065930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9a0c3-4b8b-46ac-9405-3a231c7ddeaf None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:55.066480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9a0c3-4b8b-46ac-9405-3a231c7ddeaf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:55.066480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9a0c3-4b8b-46ac-9405-3a231c7ddeaf None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:55.067863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 593/1193] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:51:55.073197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaa7df-5892-4cd1-a553-4f9d4e4c32f7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:55.073765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaa7df-5892-4cd1-a553-4f9d4e4c32f7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:55.073765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaa7df-5892-4cd1-a553-4f9d4e4c32f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:55.091775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-38eaa7df-5892-4cd1-a553-4f9d4e4c32f7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:51:55.092972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-38eaa7df-5892-4cd1-a553-4f9d4e4c32f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:51:55.136211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:55.160464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 601/1194] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:51:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:51:55.185057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9a0c3-4b8b-46ac-9405-3a231c7ddeaf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:55.285873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9a0c3-4b8b-46ac-9405-3a231c7ddeaf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:55.318713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811824cf-aaa4-4041-bfe3-3ea0f84de867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:55.319429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811824cf-aaa4-4041-bfe3-3ea0f84de867 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:55.320246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811824cf-aaa4-4041-bfe3-3ea0f84de867 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:55.320535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811824cf-aaa4-4041-bfe3-3ea0f84de867 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:55.332580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811824cf-aaa4-4041-bfe3-3ea0f84de867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-811824cf-aaa4-4041-bfe3-3ea0f84de867 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:51:55.340282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:51:55.341421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:51:55.346089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 594/1195] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:51:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:51:55.347650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaa7df-5892-4cd1-a553-4f9d4e4c32f7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:55.444562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaa7df-5892-4cd1-a553-4f9d4e4c32f7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:55.477496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3900e8-88a5-4368-9a3d-51d223421b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:55.478159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3900e8-88a5-4368-9a3d-51d223421b74 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:55.478444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3900e8-88a5-4368-9a3d-51d223421b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:55.478723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3900e8-88a5-4368-9a3d-51d223421b74 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:55.479469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3900e8-88a5-4368-9a3d-51d223421b74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:55.479779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3900e8-88a5-4368-9a3d-51d223421b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:55.481166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3900e8-88a5-4368-9a3d-51d223421b74 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:55.489109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 602/1196] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:51:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:55.495606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811824cf-aaa4-4041-bfe3-3ea0f84de867 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:55.523420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:55.597018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811824cf-aaa4-4041-bfe3-3ea0f84de867 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:55.631019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ea2da-becc-455a-bcb4-67aa52af9687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:55.631877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ea2da-becc-455a-bcb4-67aa52af9687 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:55.632182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ea2da-becc-455a-bcb4-67aa52af9687 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:55.632460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ea2da-becc-455a-bcb4-67aa52af9687 None admin] PATH_INFO: `/v3/users/cacbb3724dbd4ebeb1e6a7e9f4c0a67c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:55.633743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ea2da-becc-455a-bcb4-67aa52af9687 None admin] RBAC: Authorizing `identity:get_user(user_id=cacbb3724dbd4ebeb1e6a7e9f4c0a67c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:55.635176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ea2da-becc-455a-bcb4-67aa52af9687 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:55.636804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 595/1197] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jun 21 18:51:55 2022] GET /identity/v3/users/cacbb3724dbd4ebeb1e6a7e9f4c0a67c => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:55.642592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3900e8-88a5-4368-9a3d-51d223421b74 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:55.685551 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:55.745113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3900e8-88a5-4368-9a3d-51d223421b74 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:55.778213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c78db-5a53-4387-962a-9b163df272de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:55.778858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c78db-5a53-4387-962a-9b163df272de None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:55.779145 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c78db-5a53-4387-962a-9b163df272de None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:55.779426 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c78db-5a53-4387-962a-9b163df272de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:55.845504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c78db-5a53-4387-962a-9b163df272de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:55.846959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c78db-5a53-4387-962a-9b163df272de None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:55.886061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:55.945032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 603/1198] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:51:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:51:55.952363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ea2da-becc-455a-bcb4-67aa52af9687 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:56.045343 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:51:56.051487 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ea2da-becc-455a-bcb4-67aa52af9687 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:56.084544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39db65-32ac-48e6-8f64-6db47368e3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:56.085938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39db65-32ac-48e6-8f64-6db47368e3ab None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:56.086219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39db65-32ac-48e6-8f64-6db47368e3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:56.086847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39db65-32ac-48e6-8f64-6db47368e3ab None admin] PATH_INFO: `/v3/projects/738720f6151045b9b3b46f435e5382cf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:56.088036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39db65-32ac-48e6-8f64-6db47368e3ab None admin] RBAC: Authorizing `identity:get_project(project_id=738720f6151045b9b3b46f435e5382cf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:56.089516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39db65-32ac-48e6-8f64-6db47368e3ab None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:56.091239 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 596/1199] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jun 21 18:51:55 2022] GET /identity/v3/projects/738720f6151045b9b3b46f435e5382cf => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:51:56.097589 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c78db-5a53-4387-962a-9b163df272de None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:51:56.204976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c78db-5a53-4387-962a-9b163df272de None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:51:56.239595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3f0c0-dcc3-4ae2-9290-e25bc0ee711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:51:56.240241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3f0c0-dcc3-4ae2-9290-e25bc0ee711b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:51:56.240521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3f0c0-dcc3-4ae2-9290-e25bc0ee711b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:51:56.240798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3f0c0-dcc3-4ae2-9290-e25bc0ee711b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:51:56.242056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3f0c0-dcc3-4ae2-9290-e25bc0ee711b None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=738720f6151045b9b3b46f435e5382cf, user.id=cacbb3724dbd4ebeb1e6a7e9f4c0a67c) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:51:56.242056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3f0c0-dcc3-4ae2-9290-e25bc0ee711b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:51:56.243391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3f0c0-dcc3-4ae2-9290-e25bc0ee711b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:51:56.249199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 604/1200] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:51:56 2022] GET /identity/v3/role_assignments?user.id=cacbb3724dbd4ebeb1e6a7e9f4c0a67c&scope.project.id=738720f6151045b9b3b46f435e5382cf&role.id=e089a0d6040040c4bcf143a680488e3d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:52:06.080363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=18 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:06.246488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:52.474482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd354a-c691-4bf9-940d-8f2fa1cf6d3e None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:52.474482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd354a-c691-4bf9-940d-8f2fa1cf6d3e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:52.476990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd354a-c691-4bf9-940d-8f2fa1cf6d3e None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:52.476990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 597/1201] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:52:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:52:52.483803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d87abf-31dd-4188-b59b-4be003926424 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:52.484797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d87abf-31dd-4188-b59b-4be003926424 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:52.484797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d87abf-31dd-4188-b59b-4be003926424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:52.506826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a7d87abf-31dd-4188-b59b-4be003926424 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:52:52.508126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a7d87abf-31dd-4188-b59b-4be003926424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:52:52.608789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 605/1202] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:52:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:52:52.642761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd354a-c691-4bf9-940d-8f2fa1cf6d3e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:52.829801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd354a-c691-4bf9-940d-8f2fa1cf6d3e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:52.888125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70c7ae-3df8-433d-895c-a2aa5f504a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:52.888939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70c7ae-3df8-433d-895c-a2aa5f504a23 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:52.889270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70c7ae-3df8-433d-895c-a2aa5f504a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:52.889550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70c7ae-3df8-433d-895c-a2aa5f504a23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:52.902441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70c7ae-3df8-433d-895c-a2aa5f504a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:52.903931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70c7ae-3df8-433d-895c-a2aa5f504a23 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5e70c7ae-3df8-433d-895c-a2aa5f504a23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:52:52.933803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:52:52.935604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:52:52.937805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:52:52.937805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:52:52.937805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:52:52.937805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:52:52.937805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 598/1203] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:52:52.942717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d87abf-31dd-4188-b59b-4be003926424 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:53.078796 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d87abf-31dd-4188-b59b-4be003926424 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:53.112003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:53.123141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e3116-42b1-4105-a3cf-bcef0350b646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:53.123818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e3116-42b1-4105-a3cf-bcef0350b646 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:53.124125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e3116-42b1-4105-a3cf-bcef0350b646 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:53.124406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e3116-42b1-4105-a3cf-bcef0350b646 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:53.125135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e3116-42b1-4105-a3cf-bcef0350b646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:52:53.125459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e3116-42b1-4105-a3cf-bcef0350b646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:53.126836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e3116-42b1-4105-a3cf-bcef0350b646 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:53.139556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 606/1204] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:52:53.145390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70c7ae-3df8-433d-895c-a2aa5f504a23 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:53.210186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:53.327589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70c7ae-3df8-433d-895c-a2aa5f504a23 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:53.397001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220911-84d6-476e-bc77-cd96d0ae1608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:53.398074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220911-84d6-476e-bc77-cd96d0ae1608 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:53.398571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220911-84d6-476e-bc77-cd96d0ae1608 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:53.399048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220911-84d6-476e-bc77-cd96d0ae1608 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:53.400419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73220911-84d6-476e-bc77-cd96d0ae1608 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:53.403646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73220911-84d6-476e-bc77-cd96d0ae1608 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:53.454765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 599/1205] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jun 21 18:52:53 2022] POST /identity/v3/users => generated 251 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 18:52:55.939736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55733bad-ceda-4948-b0bf-47ae8f347601 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:55.940285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55733bad-ceda-4948-b0bf-47ae8f347601 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:55.940285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55733bad-ceda-4948-b0bf-47ae8f347601 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:55.942411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 607/1206] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:52:55.949003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fe137-1c53-4222-b9c0-b9b58c5b2163 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:55.952352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fe137-1c53-4222-b9c0-b9b58c5b2163 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:55.952352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fe137-1c53-4222-b9c0-b9b58c5b2163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:55.970130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3a3fe137-1c53-4222-b9c0-b9b58c5b2163 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:52:55.971631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3a3fe137-1c53-4222-b9c0-b9b58c5b2163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:52:56.065488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 600/1207] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:52:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:52:56.085535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55733bad-ceda-4948-b0bf-47ae8f347601 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:56.226225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55733bad-ceda-4948-b0bf-47ae8f347601 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:56.267027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f47f9-23f8-425e-a911-649ced48ae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:56.267906 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f47f9-23f8-425e-a911-649ced48ae2c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:56.268193 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f47f9-23f8-425e-a911-649ced48ae2c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:56.268477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f47f9-23f8-425e-a911-649ced48ae2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:56.281943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f47f9-23f8-425e-a911-649ced48ae2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a90f47f9-23f8-425e-a911-649ced48ae2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:52:56.290114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:52:56.291213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:52:56.292125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 608/1208] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:52:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:52:56.297691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fe137-1c53-4222-b9c0-b9b58c5b2163 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:56.356463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:56.439525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fe137-1c53-4222-b9c0-b9b58c5b2163 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:56.483721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f7470-e62c-4818-8756-aea5d715ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:56.484413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f7470-e62c-4818-8756-aea5d715ec04 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:56.484701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f7470-e62c-4818-8756-aea5d715ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:56.484980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f7470-e62c-4818-8756-aea5d715ec04 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:56.485955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f7470-e62c-4818-8756-aea5d715ec04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:52:56.486351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f7470-e62c-4818-8756-aea5d715ec04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:56.488056 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f7470-e62c-4818-8756-aea5d715ec04 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:56.498834 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 601/1209] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:52:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:52:56.504589 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f47f9-23f8-425e-a911-649ced48ae2c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:56.575784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:56.635967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f47f9-23f8-425e-a911-649ced48ae2c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:56.675580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285edba9-b3cd-4011-922d-784c3551cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:56.676357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285edba9-b3cd-4011-922d-784c3551cce1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:56.676649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285edba9-b3cd-4011-922d-784c3551cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:56.676937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285edba9-b3cd-4011-922d-784c3551cce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:56.688905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285edba9-b3cd-4011-922d-784c3551cce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:56.690446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285edba9-b3cd-4011-922d-784c3551cce1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-285edba9-b3cd-4011-922d-784c3551cce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:52:56.709324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:52:56.710432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:52:56.711359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:52:56.711359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:52:56.711359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:52:56.711359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:52:56.711359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 609/1210] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:52:56.716659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f7470-e62c-4818-8756-aea5d715ec04 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:56.840508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:56.844150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f7470-e62c-4818-8756-aea5d715ec04 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:56.891814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df553f-1d1a-49f3-a702-251ae651ef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:56.892836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df553f-1d1a-49f3-a702-251ae651ef0d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:56.893344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df553f-1d1a-49f3-a702-251ae651ef0d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:56.893830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df553f-1d1a-49f3-a702-251ae651ef0d None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:56.894776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df553f-1d1a-49f3-a702-251ae651ef0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:52:56.895656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df553f-1d1a-49f3-a702-251ae651ef0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:56.897059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df553f-1d1a-49f3-a702-251ae651ef0d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:56.915183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 602/1211] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:52:56.920895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285edba9-b3cd-4011-922d-784c3551cce1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:57.043277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285edba9-b3cd-4011-922d-784c3551cce1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:57.045401 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:57.090647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f779a1d-da03-4bcf-b273-077c714b9873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:57.091518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f779a1d-da03-4bcf-b273-077c714b9873 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:57.091867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f779a1d-da03-4bcf-b273-077c714b9873 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:57.092239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f779a1d-da03-4bcf-b273-077c714b9873 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:57.101286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f779a1d-da03-4bcf-b273-077c714b9873 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:57.102772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f779a1d-da03-4bcf-b273-077c714b9873 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-2f779a1d-da03-4bcf-b273-077c714b9873 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:52:57.121027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:52:57.123007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:52:57.124292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:52:57.125925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 610/1212] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:52:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:52:57.130105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df553f-1d1a-49f3-a702-251ae651ef0d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:57.264782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df553f-1d1a-49f3-a702-251ae651ef0d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:57.309575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208f6d9-4b2b-492a-a001-8c9362eacfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:57.310406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208f6d9-4b2b-492a-a001-8c9362eacfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:57.310836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208f6d9-4b2b-492a-a001-8c9362eacfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:57.311461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208f6d9-4b2b-492a-a001-8c9362eacfa2 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:57.312422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208f6d9-4b2b-492a-a001-8c9362eacfa2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:52:57.312742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208f6d9-4b2b-492a-a001-8c9362eacfa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:57.314251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208f6d9-4b2b-492a-a001-8c9362eacfa2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:57.337882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 603/1213] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jun 21 18:52:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:52:57.344065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f779a1d-da03-4bcf-b273-077c714b9873 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:57.484947 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f779a1d-da03-4bcf-b273-077c714b9873 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:57.536410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69740ccb-73a4-4ef7-9b2f-e9a8c1f8423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:57.537447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69740ccb-73a4-4ef7-9b2f-e9a8c1f8423d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:57.537797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69740ccb-73a4-4ef7-9b2f-e9a8c1f8423d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:57.538137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69740ccb-73a4-4ef7-9b2f-e9a8c1f8423d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:57.579762 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:57.641124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69740ccb-73a4-4ef7-9b2f-e9a8c1f8423d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:57.643671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69740ccb-73a4-4ef7-9b2f-e9a8c1f8423d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:57.777821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 611/1214] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:52:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:52:57.778848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208f6d9-4b2b-492a-a001-8c9362eacfa2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:57.801906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:57.903865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208f6d9-4b2b-492a-a001-8c9362eacfa2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:57.948331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea96d-c7d2-4f90-9347-f20d4086f8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:57.950069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea96d-c7d2-4f90-9347-f20d4086f8bb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:57.950069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea96d-c7d2-4f90-9347-f20d4086f8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:57.950069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea96d-c7d2-4f90-9347-f20d4086f8bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:57.962904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ea96d-c7d2-4f90-9347-f20d4086f8bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:57.964350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ea96d-c7d2-4f90-9347-f20d4086f8bb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-061ea96d-c7d2-4f90-9347-f20d4086f8bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:52:57.985129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:52:57.986463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:52:57.993074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:52:57.993074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:52:57.993074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:52:57.993074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:52:57.993074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 604/1215] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:52:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:52:57.994257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69740ccb-73a4-4ef7-9b2f-e9a8c1f8423d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:58.065382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:58.117748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69740ccb-73a4-4ef7-9b2f-e9a8c1f8423d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:58.162986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbf07f-6f31-488c-b8dc-070f1a9eae7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:58.163580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbf07f-6f31-488c-b8dc-070f1a9eae7f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:58.163864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbf07f-6f31-488c-b8dc-070f1a9eae7f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:58.164142 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbf07f-6f31-488c-b8dc-070f1a9eae7f None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:58.164938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbf07f-6f31-488c-b8dc-070f1a9eae7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:52:58.165282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbf07f-6f31-488c-b8dc-070f1a9eae7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:58.166717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbf07f-6f31-488c-b8dc-070f1a9eae7f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:58.180245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 612/1216] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:52:58.187614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea96d-c7d2-4f90-9347-f20d4086f8bb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:58.281016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:58.330231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea96d-c7d2-4f90-9347-f20d4086f8bb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:58.393047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745049d2-b0f6-4885-a69f-c32e3e7f1eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:58.394071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745049d2-b0f6-4885-a69f-c32e3e7f1eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:58.394506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745049d2-b0f6-4885-a69f-c32e3e7f1eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:58.394953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745049d2-b0f6-4885-a69f-c32e3e7f1eb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:58.404820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745049d2-b0f6-4885-a69f-c32e3e7f1eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:58.407702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745049d2-b0f6-4885-a69f-c32e3e7f1eb0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-745049d2-b0f6-4885-a69f-c32e3e7f1eb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:52:58.433711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:52:58.437797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:52:58.439576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:52:58.439576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:52:58.439576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:52:58.439576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:52:58.439576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:52:58.439576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:52:58.439576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:52:58.439576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:52:58.443253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 605/1217] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:52:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:52:58.444314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbf07f-6f31-488c-b8dc-070f1a9eae7f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:58.626569 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbf07f-6f31-488c-b8dc-070f1a9eae7f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:58.656478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:58.670790 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201564b9-1ae4-4157-8b44-fc753f0453ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:58.671464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201564b9-1ae4-4157-8b44-fc753f0453ef None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:58.671732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201564b9-1ae4-4157-8b44-fc753f0453ef None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:58.672011 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201564b9-1ae4-4157-8b44-fc753f0453ef None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:58.672799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201564b9-1ae4-4157-8b44-fc753f0453ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:52:58.673147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201564b9-1ae4-4157-8b44-fc753f0453ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:58.674622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201564b9-1ae4-4157-8b44-fc753f0453ef None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:58.689468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 613/1218] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:52:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:52:58.695728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745049d2-b0f6-4885-a69f-c32e3e7f1eb0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:52:58.816386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745049d2-b0f6-4885-a69f-c32e3e7f1eb0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:52:58.849231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:52:58.857230 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34eeeea-1432-46dd-a06f-e7c9afcf7ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:52:58.857683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34eeeea-1432-46dd-a06f-e7c9afcf7ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:52:58.857971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34eeeea-1432-46dd-a06f-e7c9afcf7ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:52:58.858251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34eeeea-1432-46dd-a06f-e7c9afcf7ee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:52:58.859376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34eeeea-1432-46dd-a06f-e7c9afcf7ee3 None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=f0e3fc619dc74a0591d8ad14369ae673) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:52:58.859697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34eeeea-1432-46dd-a06f-e7c9afcf7ee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:52:58.862672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34eeeea-1432-46dd-a06f-e7c9afcf7ee3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:52:58.881348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 606/1219] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:52:58 2022] GET /identity/v3/role_assignments?user.id=f0e3fc619dc74a0591d8ad14369ae673&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:00.981206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ec961-c1c3-4da8-8dd2-9322831dbbd8 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:00.981789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ec961-c1c3-4da8-8dd2-9322831dbbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:00.981902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ec961-c1c3-4da8-8dd2-9322831dbbd8 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:00.983494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 614/1220] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:53:00.989811 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334c455-752b-4a5f-ad1a-53af02cb7fa9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:00.990368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334c455-752b-4a5f-ad1a-53af02cb7fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:00.990460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334c455-752b-4a5f-ad1a-53af02cb7fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:01.011104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4334c455-752b-4a5f-ad1a-53af02cb7fa9 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:53:01.012312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4334c455-752b-4a5f-ad1a-53af02cb7fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:01.093816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 607/1221] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:53:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:53:01.118003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ec961-c1c3-4da8-8dd2-9322831dbbd8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:01.241037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ec961-c1c3-4da8-8dd2-9322831dbbd8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:01.277282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e1e4-07c0-41f6-b876-3b4962ede075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:01.278082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e1e4-07c0-41f6-b876-3b4962ede075 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:01.278369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e1e4-07c0-41f6-b876-3b4962ede075 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:01.278646 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e1e4-07c0-41f6-b876-3b4962ede075 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:01.290015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3e1e4-07c0-41f6-b876-3b4962ede075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-0de3e1e4-07c0-41f6-b876-3b4962ede075 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:53:01.301485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:01.302679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:01.303517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 615/1222] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:53:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:01.309877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334c455-752b-4a5f-ad1a-53af02cb7fa9 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:01.427031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334c455-752b-4a5f-ad1a-53af02cb7fa9 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:01.453906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:01.466697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac4df6-bc35-408a-844a-fc57490be766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:01.467371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4df6-bc35-408a-844a-fc57490be766 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:01.467659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4df6-bc35-408a-844a-fc57490be766 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:01.467974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4df6-bc35-408a-844a-fc57490be766 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:01.468705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac4df6-bc35-408a-844a-fc57490be766 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:01.469032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac4df6-bc35-408a-844a-fc57490be766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:01.470449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac4df6-bc35-408a-844a-fc57490be766 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:01.479268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 608/1223] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:53:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:01.485627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e1e4-07c0-41f6-b876-3b4962ede075 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:01.535545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:01.607855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e1e4-07c0-41f6-b876-3b4962ede075 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:01.644438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1090d94-29e2-4d59-9bbf-1fb0233476ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:01.646323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1090d94-29e2-4d59-9bbf-1fb0233476ae None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:01.646717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1090d94-29e2-4d59-9bbf-1fb0233476ae None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:01.647116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1090d94-29e2-4d59-9bbf-1fb0233476ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:01.659828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1090d94-29e2-4d59-9bbf-1fb0233476ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:01.661428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1090d94-29e2-4d59-9bbf-1fb0233476ae None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e1090d94-29e2-4d59-9bbf-1fb0233476ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:01.679350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:01.680436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:01.681743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:01.681743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:01.681743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:01.681743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:01.681743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 616/1224] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:53:01.686625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac4df6-bc35-408a-844a-fc57490be766 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:01.807597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac4df6-bc35-408a-844a-fc57490be766 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:01.844769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62718af-f2fd-4d79-8fa2-2fa581b6b772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:01.845448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62718af-f2fd-4d79-8fa2-2fa581b6b772 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:01.845773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62718af-f2fd-4d79-8fa2-2fa581b6b772 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:01.846020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62718af-f2fd-4d79-8fa2-2fa581b6b772 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:01.846832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62718af-f2fd-4d79-8fa2-2fa581b6b772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:01.847182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62718af-f2fd-4d79-8fa2-2fa581b6b772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:01.848735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62718af-f2fd-4d79-8fa2-2fa581b6b772 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:01.862459 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 609/1225] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:01.869708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1090d94-29e2-4d59-9bbf-1fb0233476ae None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:01.978299 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:01.985242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1090d94-29e2-4d59-9bbf-1fb0233476ae None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:02.031419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b5deb-ff65-4f22-a864-8d7187a9f86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:02.032736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b5deb-ff65-4f22-a864-8d7187a9f86e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:02.033300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b5deb-ff65-4f22-a864-8d7187a9f86e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:02.033773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b5deb-ff65-4f22-a864-8d7187a9f86e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:02.047070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b5deb-ff65-4f22-a864-8d7187a9f86e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:02.049772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b5deb-ff65-4f22-a864-8d7187a9f86e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d80b5deb-ff65-4f22-a864-8d7187a9f86e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:53:02.067760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:02.068822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:02.069900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 617/1226] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:02.074994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62718af-f2fd-4d79-8fa2-2fa581b6b772 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:02.129065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:02.208435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62718af-f2fd-4d79-8fa2-2fa581b6b772 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:02.256640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75dd9-ecef-4d41-b0c4-1ac6c07ab664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:02.257697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75dd9-ecef-4d41-b0c4-1ac6c07ab664 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:02.258226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75dd9-ecef-4d41-b0c4-1ac6c07ab664 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:02.258698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75dd9-ecef-4d41-b0c4-1ac6c07ab664 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:02.260280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b75dd9-ecef-4d41-b0c4-1ac6c07ab664 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:02.260280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b75dd9-ecef-4d41-b0c4-1ac6c07ab664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:02.262050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b75dd9-ecef-4d41-b0c4-1ac6c07ab664 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:02.290667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 610/1227] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jun 21 18:53:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:53:02.292134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b5deb-ff65-4f22-a864-8d7187a9f86e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:02.419915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b5deb-ff65-4f22-a864-8d7187a9f86e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:02.461534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35264243-28f6-4dcf-9060-f8ba0954b2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:02.462687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35264243-28f6-4dcf-9060-f8ba0954b2dd None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:02.463142 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35264243-28f6-4dcf-9060-f8ba0954b2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:02.463438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35264243-28f6-4dcf-9060-f8ba0954b2dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:02.476489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35264243-28f6-4dcf-9060-f8ba0954b2dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:02.477985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35264243-28f6-4dcf-9060-f8ba0954b2dd None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-35264243-28f6-4dcf-9060-f8ba0954b2dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:02.500499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:02.501596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:02.502534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:02.502534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:02.502534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:02.502534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:02.502534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 618/1228] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:02.507746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75dd9-ecef-4d41-b0c4-1ac6c07ab664 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:02.632195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75dd9-ecef-4d41-b0c4-1ac6c07ab664 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:02.634945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:02.670411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287c035-c3e3-40e3-9e62-87ea20d8e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:02.671094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c035-c3e3-40e3-9e62-87ea20d8e465 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:02.671384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c035-c3e3-40e3-9e62-87ea20d8e465 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:02.671669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287c035-c3e3-40e3-9e62-87ea20d8e465 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:02.672426 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287c035-c3e3-40e3-9e62-87ea20d8e465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:02.672734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287c035-c3e3-40e3-9e62-87ea20d8e465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:02.674271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287c035-c3e3-40e3-9e62-87ea20d8e465 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:02.693123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 611/1229] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:02.694496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35264243-28f6-4dcf-9060-f8ba0954b2dd None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:02.705541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:02.814649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35264243-28f6-4dcf-9060-f8ba0954b2dd None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:02.853837 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27fa31-6164-4360-ad61-c81fceac5118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:02.854647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27fa31-6164-4360-ad61-c81fceac5118 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:02.854983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27fa31-6164-4360-ad61-c81fceac5118 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:02.855275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27fa31-6164-4360-ad61-c81fceac5118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:02.863411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27fa31-6164-4360-ad61-c81fceac5118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:02.865154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27fa31-6164-4360-ad61-c81fceac5118 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-ab27fa31-6164-4360-ad61-c81fceac5118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:53:02.880991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:02.882169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:02.883271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:53:02.883271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:53:02.883271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:53:02.883271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:53:02.883271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:53:02.883271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:53:02.883271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:02.883271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:02.883271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 619/1230] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:53:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:53:02.888158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287c035-c3e3-40e3-9e62-87ea20d8e465 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:03.016147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287c035-c3e3-40e3-9e62-87ea20d8e465 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:03.053983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4f178-51c0-4772-a355-dd90f89860a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:03.054717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4f178-51c0-4772-a355-dd90f89860a6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:03.055005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4f178-51c0-4772-a355-dd90f89860a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:03.055286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4f178-51c0-4772-a355-dd90f89860a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:03.056236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4f178-51c0-4772-a355-dd90f89860a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:03.056632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4f178-51c0-4772-a355-dd90f89860a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:03.058263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4f178-51c0-4772-a355-dd90f89860a6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:03.073395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 612/1231] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:53:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:03.079714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27fa31-6164-4360-ad61-c81fceac5118 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:03.158328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:03.197617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27fa31-6164-4360-ad61-c81fceac5118 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:03.238068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6e623-3940-4f6d-95cf-71d0fa230658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:03.238995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6e623-3940-4f6d-95cf-71d0fa230658 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:03.239273 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6e623-3940-4f6d-95cf-71d0fa230658 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:03.239594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6e623-3940-4f6d-95cf-71d0fa230658 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/f0e3fc619dc74a0591d8ad14369ae673/roles/37aa79fddde643aeab12191e2c79681f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:03.256068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6e623-3940-4f6d-95cf-71d0fa230658 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=f0e3fc619dc74a0591d8ad14369ae673, role_id=37aa79fddde643aeab12191e2c79681f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:03.257509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6e623-3940-4f6d-95cf-71d0fa230658 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:03.279630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 620/1232] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:53:03 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/f0e3fc619dc74a0591d8ad14369ae673/roles/37aa79fddde643aeab12191e2c79681f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 18:53:05.438187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e6065-5224-4347-8a17-eba427879f40 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:05.438777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e6065-5224-4347-8a17-eba427879f40 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:05.438777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e6065-5224-4347-8a17-eba427879f40 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:05.440170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 613/1233] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:53:05.445458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279d7e4-dda1-4803-a94d-65ef79fad540 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:05.445907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279d7e4-dda1-4803-a94d-65ef79fad540 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:05.446007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279d7e4-dda1-4803-a94d-65ef79fad540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:05.462828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1279d7e4-dda1-4803-a94d-65ef79fad540 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:53:05.464032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1279d7e4-dda1-4803-a94d-65ef79fad540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:05.537222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 621/1234] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:53:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:53:05.561750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e6065-5224-4347-8a17-eba427879f40 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:05.593691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:05.683367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e6065-5224-4347-8a17-eba427879f40 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:05.724381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9a506-8d8a-4816-99b6-a3562e536457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:05.725197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9a506-8d8a-4816-99b6-a3562e536457 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:05.725530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9a506-8d8a-4816-99b6-a3562e536457 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:05.725807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9a506-8d8a-4816-99b6-a3562e536457 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:05.736840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9a506-8d8a-4816-99b6-a3562e536457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-9ab9a506-8d8a-4816-99b6-a3562e536457 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:53:05.745009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:05.746291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:05.747347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 614/1235] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jun 21 18:53:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:53:05.752703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279d7e4-dda1-4803-a94d-65ef79fad540 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:05.799160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:05.854900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279d7e4-dda1-4803-a94d-65ef79fad540 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:05.891884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b47050-932f-4e05-a926-d90357078ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:05.892552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b47050-932f-4e05-a926-d90357078ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:05.892840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b47050-932f-4e05-a926-d90357078ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:05.893144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b47050-932f-4e05-a926-d90357078ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:05.893902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b47050-932f-4e05-a926-d90357078ac4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:05.894223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b47050-932f-4e05-a926-d90357078ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:05.896226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b47050-932f-4e05-a926-d90357078ac4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:05.904838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 622/1236] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jun 21 18:53:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:05.910747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9a506-8d8a-4816-99b6-a3562e536457 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:06.002136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:06.021360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9a506-8d8a-4816-99b6-a3562e536457 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:06.057779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddeae8-3565-4cc0-b565-0f5daad912ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:06.058678 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddeae8-3565-4cc0-b565-0f5daad912ce None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:06.058963 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddeae8-3565-4cc0-b565-0f5daad912ce None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:06.059325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddeae8-3565-4cc0-b565-0f5daad912ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:06.072744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddeae8-3565-4cc0-b565-0f5daad912ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:06.074274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddeae8-3565-4cc0-b565-0f5daad912ce None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2cddeae8-3565-4cc0-b565-0f5daad912ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:06.092978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:06.094363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:06.095485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:06.095485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:06.095485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:06.095485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:06.095485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 615/1237] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:06.100213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b47050-932f-4e05-a926-d90357078ac4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:06.216292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b47050-932f-4e05-a926-d90357078ac4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:06.225878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:06.253694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb6a32-6934-43ca-91ba-6d98756ab599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:06.254892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb6a32-6934-43ca-91ba-6d98756ab599 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:06.255184 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb6a32-6934-43ca-91ba-6d98756ab599 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:06.255465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb6a32-6934-43ca-91ba-6d98756ab599 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:06.256529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb6a32-6934-43ca-91ba-6d98756ab599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:06.256884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb6a32-6934-43ca-91ba-6d98756ab599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:06.258699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb6a32-6934-43ca-91ba-6d98756ab599 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:06.273843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 623/1238] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:06.279616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddeae8-3565-4cc0-b565-0f5daad912ce None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:06.398107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddeae8-3565-4cc0-b565-0f5daad912ce None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:06.436311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10aaa77-69ed-4810-889e-e5aa012426ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:06.437136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aaa77-69ed-4810-889e-e5aa012426ab None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:06.437424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aaa77-69ed-4810-889e-e5aa012426ab None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:06.437702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aaa77-69ed-4810-889e-e5aa012426ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:06.440717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:06.447321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10aaa77-69ed-4810-889e-e5aa012426ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:06.449134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10aaa77-69ed-4810-889e-e5aa012426ab None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-f10aaa77-69ed-4810-889e-e5aa012426ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:53:06.464786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:06.466067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:06.467495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 616/1239] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:06.472368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb6a32-6934-43ca-91ba-6d98756ab599 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:06.580520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb6a32-6934-43ca-91ba-6d98756ab599 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:06.616549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbb62d-5129-4fe4-a68f-2de0d50a100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:06.617248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbb62d-5129-4fe4-a68f-2de0d50a100c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:06.617531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbb62d-5129-4fe4-a68f-2de0d50a100c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:06.617804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbb62d-5129-4fe4-a68f-2de0d50a100c None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:06.618637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbb62d-5129-4fe4-a68f-2de0d50a100c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:06.618940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbb62d-5129-4fe4-a68f-2de0d50a100c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:06.620468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbb62d-5129-4fe4-a68f-2de0d50a100c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:06.640332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 624/1240] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jun 21 18:53:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:06.646357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10aaa77-69ed-4810-889e-e5aa012426ab None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:06.756776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10aaa77-69ed-4810-889e-e5aa012426ab None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:06.793204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d1414-eb52-4593-9103-80cc8949ff2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:06.793871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d1414-eb52-4593-9103-80cc8949ff2a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:06.794162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d1414-eb52-4593-9103-80cc8949ff2a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:06.795682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d1414-eb52-4593-9103-80cc8949ff2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:06.862557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:06.870157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d1414-eb52-4593-9103-80cc8949ff2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:06.871737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d1414-eb52-4593-9103-80cc8949ff2a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:06.983666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 617/1241] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:53:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:53:06.990062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbb62d-5129-4fe4-a68f-2de0d50a100c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:07.019095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:07.110637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbb62d-5129-4fe4-a68f-2de0d50a100c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:07.156586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa333a-e678-4355-9f1b-bae8ede9a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:07.158092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa333a-e678-4355-9f1b-bae8ede9a138 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:07.158600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa333a-e678-4355-9f1b-bae8ede9a138 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:07.158953 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa333a-e678-4355-9f1b-bae8ede9a138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:07.182890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa333a-e678-4355-9f1b-bae8ede9a138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:07.186785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa333a-e678-4355-9f1b-bae8ede9a138 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-afaa333a-e678-4355-9f1b-bae8ede9a138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:07.220323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:07.221420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:07.225106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:07.225106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:07.225106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:07.225106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:07.225106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 625/1242] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:07.228152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d1414-eb52-4593-9103-80cc8949ff2a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:07.328932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:07.335020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d1414-eb52-4593-9103-80cc8949ff2a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:07.369291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4e96-6702-45ff-b292-d27c424e471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:07.369952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4e96-6702-45ff-b292-d27c424e471c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:07.370238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4e96-6702-45ff-b292-d27c424e471c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:07.370619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4e96-6702-45ff-b292-d27c424e471c None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:07.371240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4e96-6702-45ff-b292-d27c424e471c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:07.371563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4e96-6702-45ff-b292-d27c424e471c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:07.372921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4e96-6702-45ff-b292-d27c424e471c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:07.385189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 618/1243] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:07.390869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa333a-e678-4355-9f1b-bae8ede9a138 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:07.480239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:07.504259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa333a-e678-4355-9f1b-bae8ede9a138 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:07.541611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe2b68-137c-43cc-afdd-b9fe87af3e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:07.542429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe2b68-137c-43cc-afdd-b9fe87af3e82 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:07.542811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe2b68-137c-43cc-afdd-b9fe87af3e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:07.543035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe2b68-137c-43cc-afdd-b9fe87af3e82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:07.550544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe2b68-137c-43cc-afdd-b9fe87af3e82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:07.552675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe2b68-137c-43cc-afdd-b9fe87af3e82 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-aabe2b68-137c-43cc-afdd-b9fe87af3e82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:53:07.568260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:07.569387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:07.570356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:53:07.570356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:53:07.570356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:53:07.570356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:53:07.570356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:53:07.570356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:53:07.570356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:07.570356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:07.575202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 626/1244] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:53:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:07.576437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4e96-6702-45ff-b292-d27c424e471c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:07.686650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4e96-6702-45ff-b292-d27c424e471c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:07.721480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b20fbe-09b7-48c4-b7c0-364e2ff85dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:07.722138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b20fbe-09b7-48c4-b7c0-364e2ff85dbf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:07.722425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b20fbe-09b7-48c4-b7c0-364e2ff85dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:07.723212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b20fbe-09b7-48c4-b7c0-364e2ff85dbf None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:07.723998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b20fbe-09b7-48c4-b7c0-364e2ff85dbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:07.724313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b20fbe-09b7-48c4-b7c0-364e2ff85dbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:07.725855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b20fbe-09b7-48c4-b7c0-364e2ff85dbf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:07.740229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 619/1245] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:53:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:07.746819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe2b68-137c-43cc-afdd-b9fe87af3e82 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:07.860586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe2b68-137c-43cc-afdd-b9fe87af3e82 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:07.897263 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188f50-cc24-4d99-abc1-6cb2e7b63217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:07.897920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188f50-cc24-4d99-abc1-6cb2e7b63217 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:07.898206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188f50-cc24-4d99-abc1-6cb2e7b63217 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:07.898487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188f50-cc24-4d99-abc1-6cb2e7b63217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:07.899318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188f50-cc24-4d99-abc1-6cb2e7b63217 None admin] RBAC: Adding query filter params (role.id=37aa79fddde643aeab12191e2c79681f, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=f0e3fc619dc74a0591d8ad14369ae673) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:07.899631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188f50-cc24-4d99-abc1-6cb2e7b63217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:07.901146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188f50-cc24-4d99-abc1-6cb2e7b63217 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:07.908044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 627/1246] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:53:07 2022] GET /identity/v3/role_assignments?user.id=f0e3fc619dc74a0591d8ad14369ae673&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=37aa79fddde643aeab12191e2c79681f => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:10.366746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaef2d0-4255-4dd0-b2da-533b9a461134 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:10.367329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaef2d0-4255-4dd0-b2da-533b9a461134 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:10.367329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaef2d0-4255-4dd0-b2da-533b9a461134 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:10.368702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 620/1247] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:53:10.374905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf43456-e31b-4e0a-adae-6f5f96b66f3b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:10.375446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf43456-e31b-4e0a-adae-6f5f96b66f3b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:10.375446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf43456-e31b-4e0a-adae-6f5f96b66f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:10.381631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:10.395190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fdf43456-e31b-4e0a-adae-6f5f96b66f3b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:53:10.396410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fdf43456-e31b-4e0a-adae-6f5f96b66f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:10.495447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 628/1248] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:53:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:53:10.519989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaef2d0-4255-4dd0-b2da-533b9a461134 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:10.539868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:10.641524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaef2d0-4255-4dd0-b2da-533b9a461134 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:10.687019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b15c9d-ac9f-4ad2-b968-4105a2996a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:10.687962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b15c9d-ac9f-4ad2-b968-4105a2996a77 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:10.688290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b15c9d-ac9f-4ad2-b968-4105a2996a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:10.688776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b15c9d-ac9f-4ad2-b968-4105a2996a77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:10.701466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b15c9d-ac9f-4ad2-b968-4105a2996a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-b3b15c9d-ac9f-4ad2-b968-4105a2996a77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:53:10.710607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:10.711824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:10.713073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 621/1249] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:53:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:10.718576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf43456-e31b-4e0a-adae-6f5f96b66f3b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:10.854059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf43456-e31b-4e0a-adae-6f5f96b66f3b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:10.895754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:10.902461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284bf88-d1af-4111-99bc-04d73c2c7394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:10.903141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284bf88-d1af-4111-99bc-04d73c2c7394 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:10.903424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284bf88-d1af-4111-99bc-04d73c2c7394 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:10.903697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284bf88-d1af-4111-99bc-04d73c2c7394 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:10.904445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284bf88-d1af-4111-99bc-04d73c2c7394 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:10.904751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284bf88-d1af-4111-99bc-04d73c2c7394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:10.906246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284bf88-d1af-4111-99bc-04d73c2c7394 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:10.916433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 629/1250] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:53:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:53:10.922454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b15c9d-ac9f-4ad2-b968-4105a2996a77 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:11.017346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:11.048073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b15c9d-ac9f-4ad2-b968-4105a2996a77 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:11.087658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21339f4e-d2ef-42e8-ac04-8d5cf3e2e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:11.088495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21339f4e-d2ef-42e8-ac04-8d5cf3e2e419 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:11.088780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21339f4e-d2ef-42e8-ac04-8d5cf3e2e419 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:11.089060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21339f4e-d2ef-42e8-ac04-8d5cf3e2e419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:11.101353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21339f4e-d2ef-42e8-ac04-8d5cf3e2e419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:11.106205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21339f4e-d2ef-42e8-ac04-8d5cf3e2e419 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-21339f4e-d2ef-42e8-ac04-8d5cf3e2e419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:11.124362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:11.125491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:11.126466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:11.126466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:11.126466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:11.126466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:11.126466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 622/1251] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:11.132282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284bf88-d1af-4111-99bc-04d73c2c7394 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:11.254247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284bf88-d1af-4111-99bc-04d73c2c7394 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:11.295259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079c6b-84a1-40c4-b303-41322e8fa520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:11.295998 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079c6b-84a1-40c4-b303-41322e8fa520 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:11.296379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079c6b-84a1-40c4-b303-41322e8fa520 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:11.296690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079c6b-84a1-40c4-b303-41322e8fa520 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:11.297528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079c6b-84a1-40c4-b303-41322e8fa520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:11.297845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079c6b-84a1-40c4-b303-41322e8fa520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:11.299377 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079c6b-84a1-40c4-b303-41322e8fa520 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:11.314869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 630/1252] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:11.321461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21339f4e-d2ef-42e8-ac04-8d5cf3e2e419 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:11.438559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21339f4e-d2ef-42e8-ac04-8d5cf3e2e419 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:11.477335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:11.485831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e9c55-54d2-4b59-ac61-d62a5e77cc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:11.486649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e9c55-54d2-4b59-ac61-d62a5e77cc72 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:11.486936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e9c55-54d2-4b59-ac61-d62a5e77cc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:11.487217 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e9c55-54d2-4b59-ac61-d62a5e77cc72 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:11.497221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e9c55-54d2-4b59-ac61-d62a5e77cc72 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:11.498783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e9c55-54d2-4b59-ac61-d62a5e77cc72 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-b00e9c55-54d2-4b59-ac61-d62a5e77cc72 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:53:11.515072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:11.517198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:11.518551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 623/1253] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:11.523523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079c6b-84a1-40c4-b303-41322e8fa520 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:11.591956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:11.649561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079c6b-84a1-40c4-b303-41322e8fa520 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:11.692622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b96dae-3bc7-41c0-82d0-c370802d561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:11.693419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b96dae-3bc7-41c0-82d0-c370802d561c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:11.693710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b96dae-3bc7-41c0-82d0-c370802d561c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:11.694006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b96dae-3bc7-41c0-82d0-c370802d561c None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:11.694928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b96dae-3bc7-41c0-82d0-c370802d561c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:11.695281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b96dae-3bc7-41c0-82d0-c370802d561c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:11.696915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b96dae-3bc7-41c0-82d0-c370802d561c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:11.717184 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 631/1254] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jun 21 18:53:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:11.723840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e9c55-54d2-4b59-ac61-d62a5e77cc72 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:11.840685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e9c55-54d2-4b59-ac61-d62a5e77cc72 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:11.884664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee61f25-8519-4b68-bce1-f2a4b2a8b87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:11.885737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee61f25-8519-4b68-bce1-f2a4b2a8b87c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:11.886197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee61f25-8519-4b68-bce1-f2a4b2a8b87c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:11.886733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee61f25-8519-4b68-bce1-f2a4b2a8b87c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:11.966054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:11.982360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee61f25-8519-4b68-bce1-f2a4b2a8b87c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:11.985022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee61f25-8519-4b68-bce1-f2a4b2a8b87c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:12.109064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 624/1255] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:53:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:53:12.116562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b96dae-3bc7-41c0-82d0-c370802d561c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:12.237757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b96dae-3bc7-41c0-82d0-c370802d561c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:12.254560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:12.285756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6779a-882c-42f5-8a45-e2fdc2a9e2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:12.286662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6779a-882c-42f5-8a45-e2fdc2a9e2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:12.286962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6779a-882c-42f5-8a45-e2fdc2a9e2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:12.287234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6779a-882c-42f5-8a45-e2fdc2a9e2b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:12.299544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6779a-882c-42f5-8a45-e2fdc2a9e2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:12.301567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6779a-882c-42f5-8a45-e2fdc2a9e2b5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-1ae6779a-882c-42f5-8a45-e2fdc2a9e2b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:12.322062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:12.323061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:12.324484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:12.324484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:12.324484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:12.324484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:12.325260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 632/1256] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:12.330017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee61f25-8519-4b68-bce1-f2a4b2a8b87c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:12.435810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:12.451430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee61f25-8519-4b68-bce1-f2a4b2a8b87c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:12.491191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6286110-f2f2-4b5b-856a-fdf4cd2a3d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:12.491851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6286110-f2f2-4b5b-856a-fdf4cd2a3d40 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:12.492133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6286110-f2f2-4b5b-856a-fdf4cd2a3d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:12.492403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6286110-f2f2-4b5b-856a-fdf4cd2a3d40 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:12.493179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6286110-f2f2-4b5b-856a-fdf4cd2a3d40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:12.493485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6286110-f2f2-4b5b-856a-fdf4cd2a3d40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:12.494867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6286110-f2f2-4b5b-856a-fdf4cd2a3d40 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:12.509647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 625/1257] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:53:12.516726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6779a-882c-42f5-8a45-e2fdc2a9e2b5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:12.657321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6779a-882c-42f5-8a45-e2fdc2a9e2b5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:12.701519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e4c64-42d1-435f-9d6f-5a3d5ad994fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:12.702376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e4c64-42d1-435f-9d6f-5a3d5ad994fa None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:12.702783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e4c64-42d1-435f-9d6f-5a3d5ad994fa None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:12.703164 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e4c64-42d1-435f-9d6f-5a3d5ad994fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:12.714367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e4c64-42d1-435f-9d6f-5a3d5ad994fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:12.716829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e4c64-42d1-435f-9d6f-5a3d5ad994fa None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d40e4c64-42d1-435f-9d6f-5a3d5ad994fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:53:12.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:12.735093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:12.736202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:53:12.736202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:53:12.736202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:53:12.736202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:53:12.736202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:53:12.736202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:53:12.736202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:12.736202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:12.740196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 633/1258] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:53:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 18:53:12.741538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6286110-f2f2-4b5b-856a-fdf4cd2a3d40 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:12.866078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6286110-f2f2-4b5b-856a-fdf4cd2a3d40 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:12.904232 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30377eb5-772d-4831-829c-3a26c4fd7c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:12.904919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30377eb5-772d-4831-829c-3a26c4fd7c5c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:12.905245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30377eb5-772d-4831-829c-3a26c4fd7c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:12.905531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30377eb5-772d-4831-829c-3a26c4fd7c5c None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:12.906344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30377eb5-772d-4831-829c-3a26c4fd7c5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:12.906662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30377eb5-772d-4831-829c-3a26c4fd7c5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:12.908160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30377eb5-772d-4831-829c-3a26c4fd7c5c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:12.922642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 626/1259] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:53:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:12.929200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e4c64-42d1-435f-9d6f-5a3d5ad994fa None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:12.959536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:13.051008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e4c64-42d1-435f-9d6f-5a3d5ad994fa None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:13.090340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fde200-35d1-4d61-b234-b600bf5b3c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:13.091010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fde200-35d1-4d61-b234-b600bf5b3c26 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:13.091295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fde200-35d1-4d61-b234-b600bf5b3c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:13.091575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fde200-35d1-4d61-b234-b600bf5b3c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:13.092423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fde200-35d1-4d61-b234-b600bf5b3c26 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=f0e3fc619dc74a0591d8ad14369ae673) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:13.092742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fde200-35d1-4d61-b234-b600bf5b3c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:13.094268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fde200-35d1-4d61-b234-b600bf5b3c26 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:13.100661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 634/1260] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:53:12 2022] GET /identity/v3/role_assignments?user.id=f0e3fc619dc74a0591d8ad14369ae673&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=e089a0d6040040c4bcf143a680488e3d => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:15.153028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d4b08-b43c-481b-a701-c1d8ed4e7dbf None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:15.153581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d4b08-b43c-481b-a701-c1d8ed4e7dbf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:15.153675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d4b08-b43c-481b-a701-c1d8ed4e7dbf None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:15.155023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 627/1261] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:53:15.163133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e150dba-66a6-4958-960c-9f91eb150516 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:15.163627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e150dba-66a6-4958-960c-9f91eb150516 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:15.163820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e150dba-66a6-4958-960c-9f91eb150516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:15.185082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8e150dba-66a6-4958-960c-9f91eb150516 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:53:15.186740 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8e150dba-66a6-4958-960c-9f91eb150516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:15.265037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 635/1262] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:53:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:53:15.288639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d4b08-b43c-481b-a701-c1d8ed4e7dbf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:15.299653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:15.396969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d4b08-b43c-481b-a701-c1d8ed4e7dbf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:15.432954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fbc26-7e0a-4abc-b819-968e11c6f8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:15.433780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fbc26-7e0a-4abc-b819-968e11c6f8dd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:15.434069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fbc26-7e0a-4abc-b819-968e11c6f8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:15.434351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fbc26-7e0a-4abc-b819-968e11c6f8dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:15.444980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fbc26-7e0a-4abc-b819-968e11c6f8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-290fbc26-7e0a-4abc-b819-968e11c6f8dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:53:15.452685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:15.453839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:15.455029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 628/1263] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:53:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:15.460594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e150dba-66a6-4958-960c-9f91eb150516 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:15.479833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:15.560131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e150dba-66a6-4958-960c-9f91eb150516 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:15.594340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e0783-0df2-494a-b6bc-e65a4b6f46dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:15.595003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e0783-0df2-494a-b6bc-e65a4b6f46dd None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:15.595290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e0783-0df2-494a-b6bc-e65a4b6f46dd None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:15.595570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e0783-0df2-494a-b6bc-e65a4b6f46dd None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:15.596317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e0783-0df2-494a-b6bc-e65a4b6f46dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:15.596628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e0783-0df2-494a-b6bc-e65a4b6f46dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:15.598590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e0783-0df2-494a-b6bc-e65a4b6f46dd None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:15.606902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 636/1264] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:53:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:15.612630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fbc26-7e0a-4abc-b819-968e11c6f8dd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:15.679276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:15.723182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fbc26-7e0a-4abc-b819-968e11c6f8dd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:15.756924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd77f02-5c33-4a22-bf31-168779c746a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:15.757815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd77f02-5c33-4a22-bf31-168779c746a2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:15.758103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd77f02-5c33-4a22-bf31-168779c746a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:15.758383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd77f02-5c33-4a22-bf31-168779c746a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:15.769816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd77f02-5c33-4a22-bf31-168779c746a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:15.771272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd77f02-5c33-4a22-bf31-168779c746a2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6cd77f02-5c33-4a22-bf31-168779c746a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:15.788434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:15.789583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:15.790697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:15.790697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:15.790697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:15.790697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:15.790697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 629/1265] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:15.795521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e0783-0df2-494a-b6bc-e65a4b6f46dd None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:15.876928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:15.897856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e0783-0df2-494a-b6bc-e65a4b6f46dd None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:15.931097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8a594-c492-4943-ac5c-e57d900d1a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:15.931754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8a594-c492-4943-ac5c-e57d900d1a96 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:15.932037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8a594-c492-4943-ac5c-e57d900d1a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:15.932313 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8a594-c492-4943-ac5c-e57d900d1a96 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:15.933030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8a594-c492-4943-ac5c-e57d900d1a96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:15.933372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8a594-c492-4943-ac5c-e57d900d1a96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:15.934727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8a594-c492-4943-ac5c-e57d900d1a96 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:15.947352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 637/1266] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:15.953209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd77f02-5c33-4a22-bf31-168779c746a2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:16.055184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd77f02-5c33-4a22-bf31-168779c746a2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:16.075507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:16.090033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954992bd-aec7-487b-b02e-70ae66953609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:16.090830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954992bd-aec7-487b-b02e-70ae66953609 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:16.091139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954992bd-aec7-487b-b02e-70ae66953609 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:16.091416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954992bd-aec7-487b-b02e-70ae66953609 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:16.099225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954992bd-aec7-487b-b02e-70ae66953609 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:16.100656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954992bd-aec7-487b-b02e-70ae66953609 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-954992bd-aec7-487b-b02e-70ae66953609 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:53:16.121206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:16.122451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:16.123629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 630/1267] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:16.124357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8a594-c492-4943-ac5c-e57d900d1a96 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:16.245365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8a594-c492-4943-ac5c-e57d900d1a96 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:16.279398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7aef4-a4f4-4356-bb96-28ac4702c5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:16.280060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7aef4-a4f4-4356-bb96-28ac4702c5eb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:16.280343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7aef4-a4f4-4356-bb96-28ac4702c5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:16.280620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7aef4-a4f4-4356-bb96-28ac4702c5eb None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:16.281494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7aef4-a4f4-4356-bb96-28ac4702c5eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:16.281804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7aef4-a4f4-4356-bb96-28ac4702c5eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:16.283226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7aef4-a4f4-4356-bb96-28ac4702c5eb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:16.292523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:16.302019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 638/1268] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jun 21 18:53:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:16.308606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954992bd-aec7-487b-b02e-70ae66953609 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:16.412990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954992bd-aec7-487b-b02e-70ae66953609 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:16.448215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a41b9-f085-430b-b350-97731d31aa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:16.449006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a41b9-f085-430b-b350-97731d31aa11 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:16.449324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a41b9-f085-430b-b350-97731d31aa11 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:16.449606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a41b9-f085-430b-b350-97731d31aa11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:16.460780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a41b9-f085-430b-b350-97731d31aa11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:16.462274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a41b9-f085-430b-b350-97731d31aa11 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c31a41b9-f085-430b-b350-97731d31aa11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:16.480099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:16.481259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:16.482551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:16.482551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:16.482551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:16.482551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:16.482551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 631/1269] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:16.487125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7aef4-a4f4-4356-bb96-28ac4702c5eb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:16.589441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7aef4-a4f4-4356-bb96-28ac4702c5eb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:16.625098 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a3828-7469-41b8-93c2-24b08d16bf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:16.625866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a3828-7469-41b8-93c2-24b08d16bf2a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:16.626152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a3828-7469-41b8-93c2-24b08d16bf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:16.626437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a3828-7469-41b8-93c2-24b08d16bf2a None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:16.627160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a3828-7469-41b8-93c2-24b08d16bf2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:16.627470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a3828-7469-41b8-93c2-24b08d16bf2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:16.628877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a3828-7469-41b8-93c2-24b08d16bf2a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:16.642628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 639/1270] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:16.648284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a41b9-f085-430b-b350-97731d31aa11 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:16.680586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:16.755223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a41b9-f085-430b-b350-97731d31aa11 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:16.792797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1681bf-9dfe-4754-bfbb-da696925a206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:16.793628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1681bf-9dfe-4754-bfbb-da696925a206 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:16.793917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1681bf-9dfe-4754-bfbb-da696925a206 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:16.794198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1681bf-9dfe-4754-bfbb-da696925a206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:16.801223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1681bf-9dfe-4754-bfbb-da696925a206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:16.802667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1681bf-9dfe-4754-bfbb-da696925a206 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ca1681bf-9dfe-4754-bfbb-da696925a206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:53:16.817208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:16.818351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:16.819539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:53:16.819539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:53:16.819539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:53:16.819539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:53:16.819539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:53:16.819539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:53:16.819539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:16.819539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:16.819539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 632/1271] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:53:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:16.824464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a3828-7469-41b8-93c2-24b08d16bf2a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:16.883120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:16.927519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a3828-7469-41b8-93c2-24b08d16bf2a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:16.961701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e3747-6014-4783-ab05-1c5261d76cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:16.962362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e3747-6014-4783-ab05-1c5261d76cae None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:16.962650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e3747-6014-4783-ab05-1c5261d76cae None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:16.962929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e3747-6014-4783-ab05-1c5261d76cae None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:16.963708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e3747-6014-4783-ab05-1c5261d76cae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:16.964020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e3747-6014-4783-ab05-1c5261d76cae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:16.965472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e3747-6014-4783-ab05-1c5261d76cae None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:16.978349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 640/1272] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:53:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:16.984429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1681bf-9dfe-4754-bfbb-da696925a206 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:17.076470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:17.086373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1681bf-9dfe-4754-bfbb-da696925a206 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:17.124662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e060a-303a-4dce-bcd5-dac0e53f4c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:17.125606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e060a-303a-4dce-bcd5-dac0e53f4c92 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:17.125897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e060a-303a-4dce-bcd5-dac0e53f4c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:17.126194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e060a-303a-4dce-bcd5-dac0e53f4c92 None admin] PATH_INFO: `/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/f0e3fc619dc74a0591d8ad14369ae673/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:17.128887 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e060a-303a-4dce-bcd5-dac0e53f4c92 None admin] RBAC: Authorizing `identity:create_grant(project_id=0430e41e18bd4c2ebbca92c7816f8694, user_id=f0e3fc619dc74a0591d8ad14369ae673, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:17.130292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e060a-303a-4dce-bcd5-dac0e53f4c92 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:17.151272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 633/1273] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jun 21 18:53:16 2022] PUT /identity/v3/projects/0430e41e18bd4c2ebbca92c7816f8694/users/f0e3fc619dc74a0591d8ad14369ae673/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:53:19.176431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4e597-27e1-4a6d-b8a0-c894f31ba29a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:19.177038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4e597-27e1-4a6d-b8a0-c894f31ba29a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:19.177038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4e597-27e1-4a6d-b8a0-c894f31ba29a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:19.178402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 641/1274] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:53:19.184395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056adb7-85a6-47a3-95e9-ea8b102b4ff3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:19.184894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056adb7-85a6-47a3-95e9-ea8b102b4ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:19.184994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056adb7-85a6-47a3-95e9-ea8b102b4ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:19.202082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f056adb7-85a6-47a3-95e9-ea8b102b4ff3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:53:19.203305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f056adb7-85a6-47a3-95e9-ea8b102b4ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:19.272571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 634/1275] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:53:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:53:19.297128 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4e597-27e1-4a6d-b8a0-c894f31ba29a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:19.403539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4e597-27e1-4a6d-b8a0-c894f31ba29a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:19.421020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:19.440237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe63d08-2d87-42bb-858d-25b415f4703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:19.441660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe63d08-2d87-42bb-858d-25b415f4703e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:19.441939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe63d08-2d87-42bb-858d-25b415f4703e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:19.442213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe63d08-2d87-42bb-858d-25b415f4703e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:19.452558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe63d08-2d87-42bb-858d-25b415f4703e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-fbe63d08-2d87-42bb-858d-25b415f4703e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 18:53:19.460634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:19.461888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:19.462831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 642/1276] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jun 21 18:53:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:19.468287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056adb7-85a6-47a3-95e9-ea8b102b4ff3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:19.553768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:19.580113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056adb7-85a6-47a3-95e9-ea8b102b4ff3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:19.614828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57047515-acdf-4e4e-bbc4-12649c75b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:19.615502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57047515-acdf-4e4e-bbc4-12649c75b719 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:19.615788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57047515-acdf-4e4e-bbc4-12649c75b719 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:19.616068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57047515-acdf-4e4e-bbc4-12649c75b719 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:19.616825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57047515-acdf-4e4e-bbc4-12649c75b719 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:19.617166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57047515-acdf-4e4e-bbc4-12649c75b719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:19.618559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57047515-acdf-4e4e-bbc4-12649c75b719 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:19.627486 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 635/1277] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jun 21 18:53:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:19.633928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe63d08-2d87-42bb-858d-25b415f4703e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:19.750649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe63d08-2d87-42bb-858d-25b415f4703e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:19.795498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b68a64-0fcf-44f4-af12-79d1f1aceecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:19.796781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b68a64-0fcf-44f4-af12-79d1f1aceecc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:19.797069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b68a64-0fcf-44f4-af12-79d1f1aceecc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:19.797373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b68a64-0fcf-44f4-af12-79d1f1aceecc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:19.817367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b68a64-0fcf-44f4-af12-79d1f1aceecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:19.819947 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b68a64-0fcf-44f4-af12-79d1f1aceecc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-15b68a64-0fcf-44f4-af12-79d1f1aceecc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:19.844156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:19.845450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:19.846561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:19.846561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:19.846561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:19.846561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:19.846561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 643/1278] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:19.851515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57047515-acdf-4e4e-bbc4-12649c75b719 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:19.956341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57047515-acdf-4e4e-bbc4-12649c75b719 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:19.992841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cf5f2-c2f4-4d1b-86ca-39680d360d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:19.993541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cf5f2-c2f4-4d1b-86ca-39680d360d06 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:19.993827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cf5f2-c2f4-4d1b-86ca-39680d360d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:19.994104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cf5f2-c2f4-4d1b-86ca-39680d360d06 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:19.994837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856cf5f2-c2f4-4d1b-86ca-39680d360d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:19.995151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856cf5f2-c2f4-4d1b-86ca-39680d360d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:19.996539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856cf5f2-c2f4-4d1b-86ca-39680d360d06 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:20.011893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 636/1279] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 18:53:20.017760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b68a64-0fcf-44f4-af12-79d1f1aceecc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:20.053794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:20.132792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b68a64-0fcf-44f4-af12-79d1f1aceecc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:20.176040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23efb84-9f70-4d78-a46c-7ab176a06052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:20.176971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23efb84-9f70-4d78-a46c-7ab176a06052 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:20.177572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23efb84-9f70-4d78-a46c-7ab176a06052 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:20.177694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23efb84-9f70-4d78-a46c-7ab176a06052 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:20.187469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23efb84-9f70-4d78-a46c-7ab176a06052 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:20.188934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23efb84-9f70-4d78-a46c-7ab176a06052 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d23efb84-9f70-4d78-a46c-7ab176a06052 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 18:53:20.207808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:20.209061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:20.210182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 644/1280] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:20.215822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cf5f2-c2f4-4d1b-86ca-39680d360d06 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:20.219952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:20.319869 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cf5f2-c2f4-4d1b-86ca-39680d360d06 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:20.357020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee1c9c-9e9a-4b7e-a1eb-e93b454729dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:20.357725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee1c9c-9e9a-4b7e-a1eb-e93b454729dc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:20.358015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee1c9c-9e9a-4b7e-a1eb-e93b454729dc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:20.358298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee1c9c-9e9a-4b7e-a1eb-e93b454729dc None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:20.359148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee1c9c-9e9a-4b7e-a1eb-e93b454729dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:20.359462 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee1c9c-9e9a-4b7e-a1eb-e93b454729dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:20.360912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee1c9c-9e9a-4b7e-a1eb-e93b454729dc None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:20.380146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 637/1281] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jun 21 18:53:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:20.387257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23efb84-9f70-4d78-a46c-7ab176a06052 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:20.481291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:20.493775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23efb84-9f70-4d78-a46c-7ab176a06052 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:20.527329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3a4bd-313c-438d-8a4a-ce73400dbe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:20.527981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3a4bd-313c-438d-8a4a-ce73400dbe48 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:20.528267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3a4bd-313c-438d-8a4a-ce73400dbe48 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:20.528548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3a4bd-313c-438d-8a4a-ce73400dbe48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:20.598051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3a4bd-313c-438d-8a4a-ce73400dbe48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:20.599419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3a4bd-313c-438d-8a4a-ce73400dbe48 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:20.687597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:20.698614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 645/1282] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jun 21 18:53:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 18:53:20.704586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee1c9c-9e9a-4b7e-a1eb-e93b454729dc None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:20.782516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:20.821739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee1c9c-9e9a-4b7e-a1eb-e93b454729dc None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:20.857146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a467c-f3c7-43f2-b363-83684b43f802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:20.857855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a467c-f3c7-43f2-b363-83684b43f802 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:20.858144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a467c-f3c7-43f2-b363-83684b43f802 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:20.858424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a467c-f3c7-43f2-b363-83684b43f802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:20.868649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a467c-f3c7-43f2-b363-83684b43f802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:20.870132 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a467c-f3c7-43f2-b363-83684b43f802 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2d6a467c-f3c7-43f2-b363-83684b43f802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 18:53:20.888165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:20.889137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:20.890172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 18:53:20.890172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 18:53:20.890172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 18:53:20.890172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:20.890172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 638/1283] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:20.898204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3a4bd-313c-438d-8a4a-ce73400dbe48 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:20.998474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3a4bd-313c-438d-8a4a-ce73400dbe48 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:21.033725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ab912-de52-49e4-a00b-f7ace227929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:21.034418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ab912-de52-49e4-a00b-f7ace227929f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:21.034708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ab912-de52-49e4-a00b-f7ace227929f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:21.034987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ab912-de52-49e4-a00b-f7ace227929f None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:21.035744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ab912-de52-49e4-a00b-f7ace227929f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:21.036055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ab912-de52-49e4-a00b-f7ace227929f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:21.037516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ab912-de52-49e4-a00b-f7ace227929f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:21.049188 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 646/1284] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jun 21 18:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:21.054813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a467c-f3c7-43f2-b363-83684b43f802 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:21.161127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a467c-f3c7-43f2-b363-83684b43f802 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:21.175549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:21.197106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b058c2-9673-4e3e-aac9-efa7e31fa970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:21.197909 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b058c2-9673-4e3e-aac9-efa7e31fa970 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:21.198202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b058c2-9673-4e3e-aac9-efa7e31fa970 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:21.198479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b058c2-9673-4e3e-aac9-efa7e31fa970 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:21.205328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b058c2-9673-4e3e-aac9-efa7e31fa970 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:21.207368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b058c2-9673-4e3e-aac9-efa7e31fa970 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c5b058c2-9673-4e3e-aac9-efa7e31fa970 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 18:53:21.221108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:21.222629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:21.223747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 18:53:21.223747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 18:53:21.223747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 18:53:21.223747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 18:53:21.223747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 18:53:21.223747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 18:53:21.223747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 18:53:21.223747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 21 18:53:21.223747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 639/1285] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jun 21 18:53:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:21.228397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ab912-de52-49e4-a00b-f7ace227929f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:21.279282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:21.332220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ab912-de52-49e4-a00b-f7ace227929f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:21.366497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea97fce-55e4-46ea-91a5-5359dcd4bdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:21.367153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea97fce-55e4-46ea-91a5-5359dcd4bdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:21.367436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea97fce-55e4-46ea-91a5-5359dcd4bdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:21.367710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea97fce-55e4-46ea-91a5-5359dcd4bdc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:21.368491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea97fce-55e4-46ea-91a5-5359dcd4bdc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:21.368797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea97fce-55e4-46ea-91a5-5359dcd4bdc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:21.370244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea97fce-55e4-46ea-91a5-5359dcd4bdc4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:21.382954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 647/1286] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jun 21 18:53:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:21.389380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b058c2-9673-4e3e-aac9-efa7e31fa970 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:21.490520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b058c2-9673-4e3e-aac9-efa7e31fa970 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:21.524448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e87769-b185-48a5-baf2-c12dcb78e156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:21.525158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e87769-b185-48a5-baf2-c12dcb78e156 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:21.525451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e87769-b185-48a5-baf2-c12dcb78e156 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:21.525735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e87769-b185-48a5-baf2-c12dcb78e156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:21.526577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e87769-b185-48a5-baf2-c12dcb78e156 None admin] RBAC: Adding query filter params (role.id=e089a0d6040040c4bcf143a680488e3d, scope.project.id=0430e41e18bd4c2ebbca92c7816f8694, user.id=f0e3fc619dc74a0591d8ad14369ae673) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:21.526897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e87769-b185-48a5-baf2-c12dcb78e156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:21.528346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e87769-b185-48a5-baf2-c12dcb78e156 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:21.534569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 640/1287] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jun 21 18:53:21 2022] GET /identity/v3/role_assignments?user.id=f0e3fc619dc74a0591d8ad14369ae673&scope.project.id=0430e41e18bd4c2ebbca92c7816f8694&role.id=e089a0d6040040c4bcf143a680488e3d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:23.623649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d482-4b19-480d-81f9-03298e204749 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:23.624166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d482-4b19-480d-81f9-03298e204749 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:23.624166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d482-4b19-480d-81f9-03298e204749 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:23.625550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 648/1288] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:53:23.633625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853ade-ba99-4293-9d39-4810f831de4d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:23.634121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853ade-ba99-4293-9d39-4810f831de4d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:23.634526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853ade-ba99-4293-9d39-4810f831de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:23.651410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a2853ade-ba99-4293-9d39-4810f831de4d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:53:23.652611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a2853ade-ba99-4293-9d39-4810f831de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:23.690569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:23.730294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 641/1289] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:53:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:53:23.753891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5d482-4b19-480d-81f9-03298e204749 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:23.857106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5d482-4b19-480d-81f9-03298e204749 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:23.868811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:23.893381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07c13f-2a02-437b-8e9a-7508020fcc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:23.894123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07c13f-2a02-437b-8e9a-7508020fcc47 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:23.894410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07c13f-2a02-437b-8e9a-7508020fcc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:23.894688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07c13f-2a02-437b-8e9a-7508020fcc47 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:23.895346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07c13f-2a02-437b-8e9a-7508020fcc47 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:23.896722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07c13f-2a02-437b-8e9a-7508020fcc47 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5d07c13f-2a02-437b-8e9a-7508020fcc47 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:23.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:23.911167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:23.912968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 649/1290] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:53:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:23.916880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853ade-ba99-4293-9d39-4810f831de4d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:24.019526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853ade-ba99-4293-9d39-4810f831de4d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:24.053716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a5ad0-3bff-4472-808b-d3068d5ab384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:24.054378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a5ad0-3bff-4472-808b-d3068d5ab384 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:24.054661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a5ad0-3bff-4472-808b-d3068d5ab384 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:24.054936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a5ad0-3bff-4472-808b-d3068d5ab384 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:24.055639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a5ad0-3bff-4472-808b-d3068d5ab384 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:24.055944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a5ad0-3bff-4472-808b-d3068d5ab384 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:24.057345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a5ad0-3bff-4472-808b-d3068d5ab384 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:24.062743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 642/1291] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Jun 21 18:53:23 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:24.068612 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07c13f-2a02-437b-8e9a-7508020fcc47 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:24.180823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07c13f-2a02-437b-8e9a-7508020fcc47 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:24.217874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76adb76-c421-41f7-833c-4616898b3674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:24.218534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76adb76-c421-41f7-833c-4616898b3674 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:24.218817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76adb76-c421-41f7-833c-4616898b3674 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:24.219095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76adb76-c421-41f7-833c-4616898b3674 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:24.219797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76adb76-c421-41f7-833c-4616898b3674 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:24.220102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76adb76-c421-41f7-833c-4616898b3674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:24.221494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76adb76-c421-41f7-833c-4616898b3674 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:24.226641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 650/1292] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Jun 21 18:53:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:26.265930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75c6f3-5b5a-4df5-8cc5-5bcf658948ca None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:26.266491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75c6f3-5b5a-4df5-8cc5-5bcf658948ca None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:26.266491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75c6f3-5b5a-4df5-8cc5-5bcf658948ca None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:26.267874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 643/1293] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:53:26.273935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b10030-9ba0-4d06-ac2b-e544e62b5463 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:26.274402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b10030-9ba0-4d06-ac2b-e544e62b5463 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:26.274503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b10030-9ba0-4d06-ac2b-e544e62b5463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:26.291238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-01b10030-9ba0-4d06-ac2b-e544e62b5463 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:53:26.292427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-01b10030-9ba0-4d06-ac2b-e544e62b5463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:26.325943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:26.364948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 651/1294] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:53:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:53:26.388003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75c6f3-5b5a-4df5-8cc5-5bcf658948ca None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:26.419595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:26.492469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75c6f3-5b5a-4df5-8cc5-5bcf658948ca None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:26.527090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ead3b-ac46-405f-bfa7-7e130ddf36fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:26.527662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ead3b-ac46-405f-bfa7-7e130ddf36fe None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:26.527945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ead3b-ac46-405f-bfa7-7e130ddf36fe None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:26.528221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ead3b-ac46-405f-bfa7-7e130ddf36fe None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:26.528972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ead3b-ac46-405f-bfa7-7e130ddf36fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:26.530386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ead3b-ac46-405f-bfa7-7e130ddf36fe None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:26.543772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 644/1295] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jun 21 18:53:26 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:53:28.578124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f562a9b-442d-4ae1-bb8f-5c92924ecfa6 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:28.578626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f562a9b-442d-4ae1-bb8f-5c92924ecfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:28.578626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f562a9b-442d-4ae1-bb8f-5c92924ecfa6 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:28.579965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 652/1296] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:53:28.585416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6002c-dc53-4e9d-9a54-a8cd985278c4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:28.585855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6002c-dc53-4e9d-9a54-a8cd985278c4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:28.585980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6002c-dc53-4e9d-9a54-a8cd985278c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:28.603244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-38e6002c-dc53-4e9d-9a54-a8cd985278c4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:53:28.604444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-38e6002c-dc53-4e9d-9a54-a8cd985278c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:28.684021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 645/1297] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:53:28 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:53:28.707492 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f562a9b-442d-4ae1-bb8f-5c92924ecfa6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:28.818042 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f562a9b-442d-4ae1-bb8f-5c92924ecfa6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:28.852490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9a286-17d9-4d4b-937a-fef7c8835584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:28.853242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9a286-17d9-4d4b-937a-fef7c8835584 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:28.853527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9a286-17d9-4d4b-937a-fef7c8835584 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:28.853803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9a286-17d9-4d4b-937a-fef7c8835584 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:28.854460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9a286-17d9-4d4b-937a-fef7c8835584 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:28.855833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9a286-17d9-4d4b-937a-fef7c8835584 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-adf9a286-17d9-4d4b-937a-fef7c8835584 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:28.867901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:28.869216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:28.870298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 653/1298] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:53:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:28.875226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6002c-dc53-4e9d-9a54-a8cd985278c4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:28.894104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:28.977068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6002c-dc53-4e9d-9a54-a8cd985278c4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:29.010522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdc5f2-dd39-43bf-a7b4-2ea4d4b59987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:29.011194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdc5f2-dd39-43bf-a7b4-2ea4d4b59987 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:29.011480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdc5f2-dd39-43bf-a7b4-2ea4d4b59987 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:29.011758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdc5f2-dd39-43bf-a7b4-2ea4d4b59987 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:29.012474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdc5f2-dd39-43bf-a7b4-2ea4d4b59987 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:29.012785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdc5f2-dd39-43bf-a7b4-2ea4d4b59987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:29.014202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdc5f2-dd39-43bf-a7b4-2ea4d4b59987 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:29.020623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 646/1299] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Jun 21 18:53:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:29.026573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9a286-17d9-4d4b-937a-fef7c8835584 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:29.058398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:29.141902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9a286-17d9-4d4b-937a-fef7c8835584 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:29.178236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf2cf-e6a8-4d70-b4a5-f4213135cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:29.178889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf2cf-e6a8-4d70-b4a5-f4213135cb87 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:29.179171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf2cf-e6a8-4d70-b4a5-f4213135cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:29.179446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf2cf-e6a8-4d70-b4a5-f4213135cb87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:29.180259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf2cf-e6a8-4d70-b4a5-f4213135cb87 None admin] RBAC: Adding query filter params (interface=public, service_id=721c2046e13b42a698729534c7768fe6, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:29.180569 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf2cf-e6a8-4d70-b4a5-f4213135cb87 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:29.181955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf2cf-e6a8-4d70-b4a5-f4213135cb87 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:29.187438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 654/1300] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jun 21 18:53:29 2022] GET /identity/v3/endpoints?service_id=721c2046e13b42a698729534c7768fe6&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:29.193748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdc5f2-dd39-43bf-a7b4-2ea4d4b59987 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:29.286504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:29.304625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdc5f2-dd39-43bf-a7b4-2ea4d4b59987 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:29.341513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b011c1-c7a6-496f-8627-801c3e9a94ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:29.342196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b011c1-c7a6-496f-8627-801c3e9a94ac None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:29.342478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b011c1-c7a6-496f-8627-801c3e9a94ac None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:29.342750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b011c1-c7a6-496f-8627-801c3e9a94ac None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:29.343459 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b011c1-c7a6-496f-8627-801c3e9a94ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:29.344864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b011c1-c7a6-496f-8627-801c3e9a94ac None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:29.351861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 647/1301] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jun 21 18:53:29 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:53:31.381382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58df13c-3b15-47de-b7a4-7af4a9e83e6c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:31.381930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58df13c-3b15-47de-b7a4-7af4a9e83e6c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:31.381930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58df13c-3b15-47de-b7a4-7af4a9e83e6c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:31.383324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 655/1302] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:53:31.388848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84511-70b8-4522-a3ee-99c153342492 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:31.389305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84511-70b8-4522-a3ee-99c153342492 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:31.389449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84511-70b8-4522-a3ee-99c153342492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:31.407738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d7e84511-70b8-4522-a3ee-99c153342492 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:53:31.409559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d7e84511-70b8-4522-a3ee-99c153342492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:53:31.477664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 648/1303] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:53:31 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 18:53:31.501839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58df13c-3b15-47de-b7a4-7af4a9e83e6c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:31.598831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:31.606030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58df13c-3b15-47de-b7a4-7af4a9e83e6c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:31.646961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b33ee-d850-42b6-9b0f-f88b708cba85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:31.647753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b33ee-d850-42b6-9b0f-f88b708cba85 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:31.648035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b33ee-d850-42b6-9b0f-f88b708cba85 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:31.648311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b33ee-d850-42b6-9b0f-f88b708cba85 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:31.648959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b33ee-d850-42b6-9b0f-f88b708cba85 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:31.650353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b33ee-d850-42b6-9b0f-f88b708cba85 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7d9b33ee-d850-42b6-9b0f-f88b708cba85 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 18:53:31.662822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 18:53:31.663894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jun 21 18:53:31.664848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 656/1304] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jun 21 18:53:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 18:53:31.672186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e84511-70b8-4522-a3ee-99c153342492 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:31.751173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:31.774731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e84511-70b8-4522-a3ee-99c153342492 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:31.808726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016181c-45e2-4ce5-904a-19a9e5193f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:31.809423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016181c-45e2-4ce5-904a-19a9e5193f6d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:31.809709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016181c-45e2-4ce5-904a-19a9e5193f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:31.809988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016181c-45e2-4ce5-904a-19a9e5193f6d None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:31.810990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016181c-45e2-4ce5-904a-19a9e5193f6d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:53:31.811298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016181c-45e2-4ce5-904a-19a9e5193f6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:31.812675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016181c-45e2-4ce5-904a-19a9e5193f6d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:31.818035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 649/1305] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Jun 21 18:53:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:53:31.823797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b33ee-d850-42b6-9b0f-f88b708cba85 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:53:31.926624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b33ee-d850-42b6-9b0f-f88b708cba85 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:53:31.961910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a151a-adfa-4461-8b64-a2d3a8f9fbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:53:31.962572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a151a-adfa-4461-8b64-a2d3a8f9fbc0 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:53:31.962868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a151a-adfa-4461-8b64-a2d3a8f9fbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:53:31.963150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a151a-adfa-4461-8b64-a2d3a8f9fbc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:53:31.963890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a151a-adfa-4461-8b64-a2d3a8f9fbc0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:53:31.965291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a151a-adfa-4461-8b64-a2d3a8f9fbc0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:53:31.985340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 657/1306] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jun 21 18:53:31 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:53:41.814947 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=30 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:53:41.982969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:35.747746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b0bd0-5b5e-44e6-ac93-123150d99f5f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:35.748326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b0bd0-5b5e-44e6-ac93-123150d99f5f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:35.748326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b0bd0-5b5e-44e6-ac93-123150d99f5f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:35.755564 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90b87d-eb87-4e97-afe6-ba6ecd3129fb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:35.756033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90b87d-eb87-4e97-afe6-ba6ecd3129fb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:35.756125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90b87d-eb87-4e97-afe6-ba6ecd3129fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:35.759589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 650/1307] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:54:35 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:35.846950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fd90b87d-eb87-4e97-afe6-ba6ecd3129fb None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:35.848183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fd90b87d-eb87-4e97-afe6-ba6ecd3129fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:36.087550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 658/1308] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:54:36.123623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda232b0-8e9b-4784-a1d1-1c120c5ea415 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:36.124109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda232b0-8e9b-4784-a1d1-1c120c5ea415 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:36.124350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda232b0-8e9b-4784-a1d1-1c120c5ea415 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:36.126602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 651/1309] 10.222.0.70 () {54 vars in 964 bytes} [Tue Jun 21 18:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:36.133703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0488b4a-cedb-4880-a193-01b7c704cc87 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:36.133997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0488b4a-cedb-4880-a193-01b7c704cc87 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:36.134276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0488b4a-cedb-4880-a193-01b7c704cc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:36.235766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d0488b4a-cedb-4880-a193-01b7c704cc87 None None] MFA Rules not processed for user `808b530d24b3487b9a20ee74c131502e`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:36.236973 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d0488b4a-cedb-4880-a193-01b7c704cc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:36.259888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:36.332712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 659/1310] 10.222.0.70 () {58 vars in 1061 bytes} [Tue Jun 21 18:54:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:54:36.343030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda232b0-8e9b-4784-a1d1-1c120c5ea415 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:54:36.345849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-dda232b0-8e9b-4784-a1d1-1c120c5ea415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:36.524589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda232b0-8e9b-4784-a1d1-1c120c5ea415 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:54:36.610669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f734eb8-a310-4d89-a4bc-622d717962ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:54:36.611333 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f734eb8-a310-4d89-a4bc-622d717962ec service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:36.611675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f734eb8-a310-4d89-a4bc-622d717962ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:36.611959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f734eb8-a310-4d89-a4bc-622d717962ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:36.613874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1f734eb8-a310-4d89-a4bc-622d717962ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:36.691317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:36.754981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f734eb8-a310-4d89-a4bc-622d717962ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:54:36.756444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f734eb8-a310-4d89-a4bc-622d717962ec service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:54:36.779028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 652/1311] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:54:39.194106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6dab4-ce42-4dbf-8fdc-ee53ffbe0923 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:39.194563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6dab4-ce42-4dbf-8fdc-ee53ffbe0923 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:39.194853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6dab4-ce42-4dbf-8fdc-ee53ffbe0923 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:39.197262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 660/1312] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:54:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:39.204193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9fbf-293d-4138-9f07-95944c658568 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:39.204663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9fbf-293d-4138-9f07-95944c658568 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:39.204943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9fbf-293d-4138-9f07-95944c658568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:39.287734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-300b9fbf-293d-4138-9f07-95944c658568 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:39.288956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-300b9fbf-293d-4138-9f07-95944c658568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:39.401275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 653/1313] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:54:39.452164 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29853e19-1908-4716-be30-3f0ccbbfc135 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:39.452402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29853e19-1908-4716-be30-3f0ccbbfc135 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:39.452657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29853e19-1908-4716-be30-3f0ccbbfc135 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:39.454087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 661/1314] 10.222.0.70 () {54 vars in 964 bytes} [Tue Jun 21 18:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:39.459741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf519b4-881d-4a14-aaff-6f972e9adac8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:39.460025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf519b4-881d-4a14-aaff-6f972e9adac8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:39.460301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf519b4-881d-4a14-aaff-6f972e9adac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:39.470690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:39.534655 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-bcf519b4-881d-4a14-aaff-6f972e9adac8 None None] MFA Rules not processed for user `808b530d24b3487b9a20ee74c131502e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:39.535854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bcf519b4-881d-4a14-aaff-6f972e9adac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:39.616764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 654/1315] 10.222.0.70 () {58 vars in 1061 bytes} [Tue Jun 21 18:54:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:54:39.626750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29853e19-1908-4716-be30-3f0ccbbfc135 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:54:39.628859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-29853e19-1908-4716-be30-3f0ccbbfc135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:39.745833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29853e19-1908-4716-be30-3f0ccbbfc135 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:54:39.820682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:39.823241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e1e21-59fd-4eaf-808c-f244b84e866b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:54:39.824104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e1e21-59fd-4eaf-808c-f244b84e866b service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:39.824460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e1e21-59fd-4eaf-808c-f244b84e866b service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:39.824909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e1e21-59fd-4eaf-808c-f244b84e866b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:39.827001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-092e1e21-59fd-4eaf-808c-f244b84e866b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:39.945530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e1e21-59fd-4eaf-808c-f244b84e866b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:54:39.947286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e1e21-59fd-4eaf-808c-f244b84e866b service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:54:39.966841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 662/1316] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:54:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:54:42.548244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8eb34-624f-46db-b318-0c7370b5471f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:42.548777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8eb34-624f-46db-b318-0c7370b5471f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:42.548777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8eb34-624f-46db-b318-0c7370b5471f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:42.550200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 655/1317] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:42.555250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdaeff-cde8-4958-82cb-726076747ddd None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:42.555647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdaeff-cde8-4958-82cb-726076747ddd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:42.555808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdaeff-cde8-4958-82cb-726076747ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:42.619300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8abdaeff-cde8-4958-82cb-726076747ddd None None] MFA Rules not processed for user `d809509539f742ddb1d244a1c6f9e2af`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:42.620400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8abdaeff-cde8-4958-82cb-726076747ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:42.670832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:42.715260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 663/1318] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:54:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:54:42.759498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8eb34-624f-46db-b318-0c7370b5471f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:54:42.761056 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3ae8eb34-624f-46db-b318-0c7370b5471f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:42.865174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8eb34-624f-46db-b318-0c7370b5471f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:54:42.867979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:42.939009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276cb03-6fbe-4b9f-8ddb-963cdaadd3df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:54:42.939650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276cb03-6fbe-4b9f-8ddb-963cdaadd3df service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:42.939935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276cb03-6fbe-4b9f-8ddb-963cdaadd3df service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:42.940213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276cb03-6fbe-4b9f-8ddb-963cdaadd3df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:42.941892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6276cb03-6fbe-4b9f-8ddb-963cdaadd3df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:43.049167 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276cb03-6fbe-4b9f-8ddb-963cdaadd3df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:54:43.050610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276cb03-6fbe-4b9f-8ddb-963cdaadd3df service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:54:43.069081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 656/1319] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:54:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:54:45.656402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a45a57-10aa-4ad8-93ef-e519f534fc8c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:45.656914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a45a57-10aa-4ad8-93ef-e519f534fc8c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:45.656914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a45a57-10aa-4ad8-93ef-e519f534fc8c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:45.658300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 664/1320] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:45.663409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fe136-931c-4005-8136-df5402840ed9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:45.663788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fe136-931c-4005-8136-df5402840ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:45.663965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fe136-931c-4005-8136-df5402840ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:45.704008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:45.730075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-af9fe136-931c-4005-8136-df5402840ed9 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:45.731281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-af9fe136-931c-4005-8136-df5402840ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:45.832166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 657/1321] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:54:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:54:45.871278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a45a57-10aa-4ad8-93ef-e519f534fc8c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:54:45.872856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d3a45a57-10aa-4ad8-93ef-e519f534fc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:45.975935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a45a57-10aa-4ad8-93ef-e519f534fc8c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:54:46.047157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584ca32-23ad-4f73-8312-1b5009f6d574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:54:46.047794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584ca32-23ad-4f73-8312-1b5009f6d574 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:46.048074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584ca32-23ad-4f73-8312-1b5009f6d574 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:46.048348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584ca32-23ad-4f73-8312-1b5009f6d574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:46.050010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2584ca32-23ad-4f73-8312-1b5009f6d574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:46.057378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:46.165723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584ca32-23ad-4f73-8312-1b5009f6d574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:54:46.167182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584ca32-23ad-4f73-8312-1b5009f6d574 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:54:46.187949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 665/1322] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:54:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:54:48.908969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0ac44-eeba-44ab-96e8-9807f2207556 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:48.909571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0ac44-eeba-44ab-96e8-9807f2207556 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:48.909571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0ac44-eeba-44ab-96e8-9807f2207556 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:48.910962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 658/1323] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:48.916440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827ff3a-d230-4762-aad8-7a9b14f872d5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:48.916910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827ff3a-d230-4762-aad8-7a9b14f872d5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:48.917045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827ff3a-d230-4762-aad8-7a9b14f872d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:48.988041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1827ff3a-d230-4762-aad8-7a9b14f872d5 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:48.989304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1827ff3a-d230-4762-aad8-7a9b14f872d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:49.050826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:49.096295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 666/1324] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:54:49.134806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0ac44-eeba-44ab-96e8-9807f2207556 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:54:49.136399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1ba0ac44-eeba-44ab-96e8-9807f2207556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:49.253077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0ac44-eeba-44ab-96e8-9807f2207556 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:54:49.266589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:49.340502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af274ae6-ba38-4fa2-9f24-8ad1e46b9074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:54:49.341170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274ae6-ba38-4fa2-9f24-8ad1e46b9074 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:49.341452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274ae6-ba38-4fa2-9f24-8ad1e46b9074 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:49.341726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274ae6-ba38-4fa2-9f24-8ad1e46b9074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:49.343482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-af274ae6-ba38-4fa2-9f24-8ad1e46b9074 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:49.456710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af274ae6-ba38-4fa2-9f24-8ad1e46b9074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:54:49.458207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af274ae6-ba38-4fa2-9f24-8ad1e46b9074 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:54:49.476700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 659/1325] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:54:51.628509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b99f-9012-4334-9010-b1e0f0e8ec23 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:51.628953 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b99f-9012-4334-9010-b1e0f0e8ec23 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:51.629041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b99f-9012-4334-9010-b1e0f0e8ec23 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:51.631046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 667/1326] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:51.637229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e880-37d4-488a-b6f8-8d0c3f00eca3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:51.638046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e880-37d4-488a-b6f8-8d0c3f00eca3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:51.638046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e880-37d4-488a-b6f8-8d0c3f00eca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:51.703254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:51.726671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-83d8e880-37d4-488a-b6f8-8d0c3f00eca3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:51.727891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-83d8e880-37d4-488a-b6f8-8d0c3f00eca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:51.834780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 660/1327] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:54:51.874063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0b99f-9012-4334-9010-b1e0f0e8ec23 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:54:51.876235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d5e0b99f-9012-4334-9010-b1e0f0e8ec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:52.006566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0b99f-9012-4334-9010-b1e0f0e8ec23 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:54:52.091127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8f788-49b7-4e7e-a35d-8190d8691ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:54:52.091777 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8f788-49b7-4e7e-a35d-8190d8691ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:52.092062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8f788-49b7-4e7e-a35d-8190d8691ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:52.092347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8f788-49b7-4e7e-a35d-8190d8691ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:52.094637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-52a8f788-49b7-4e7e-a35d-8190d8691ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:52.113623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:52.217226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8f788-49b7-4e7e-a35d-8190d8691ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:54:52.219296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8f788-49b7-4e7e-a35d-8190d8691ec1 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:54:52.239552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 668/1328] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:54:54.350876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4cfb6-2d2d-4872-b3ae-e2e0964e7eae None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:54.351424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4cfb6-2d2d-4872-b3ae-e2e0964e7eae None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:54.351424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4cfb6-2d2d-4872-b3ae-e2e0964e7eae None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:54.352830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 661/1329] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:54.358273 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a047d8a-5241-4713-8c42-7d45861f4749 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:54.358744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a047d8a-5241-4713-8c42-7d45861f4749 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:54.358859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a047d8a-5241-4713-8c42-7d45861f4749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:54.428956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3a047d8a-5241-4713-8c42-7d45861f4749 None None] MFA Rules not processed for user `d809509539f742ddb1d244a1c6f9e2af`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:54.430215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3a047d8a-5241-4713-8c42-7d45861f4749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:54.501231 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:54.533913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 669/1330] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:54:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:54:54.572293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4cfb6-2d2d-4872-b3ae-e2e0964e7eae None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:54:54.573967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-91e4cfb6-2d2d-4872-b3ae-e2e0964e7eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:54.681587 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4cfb6-2d2d-4872-b3ae-e2e0964e7eae None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:54:54.703495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:54.758237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f836d-252c-4727-81a0-bd2d40688241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:54:54.758893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f836d-252c-4727-81a0-bd2d40688241 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:54.759182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f836d-252c-4727-81a0-bd2d40688241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:54.759479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f836d-252c-4727-81a0-bd2d40688241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:54.761178 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-480f836d-252c-4727-81a0-bd2d40688241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:54.898843 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f836d-252c-4727-81a0-bd2d40688241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:54:54.900313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f836d-252c-4727-81a0-bd2d40688241 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:54:54.920709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 662/1331] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:54:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:54:57.414462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940009b-e195-428e-9634-3a558f9aebd5 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:57.414924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940009b-e195-428e-9634-3a558f9aebd5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:57.415031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940009b-e195-428e-9634-3a558f9aebd5 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:57.416437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 670/1332] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:54:57.422526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc32af-f4a8-4818-b3b3-f38df5580357 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:57.423014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc32af-f4a8-4818-b3b3-f38df5580357 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:57.423108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc32af-f4a8-4818-b3b3-f38df5580357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:57.488025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:57.491434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-70cc32af-f4a8-4818-b3b3-f38df5580357 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:54:57.492874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-70cc32af-f4a8-4818-b3b3-f38df5580357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:57.593797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 663/1333] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:54:57.630219 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940009b-e195-428e-9634-3a558f9aebd5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:54:57.631759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4940009b-e195-428e-9634-3a558f9aebd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:57.736676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940009b-e195-428e-9634-3a558f9aebd5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:54:57.809048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dd248-3c6b-4abe-b2b1-153dc2405640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:54:57.809726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dd248-3c6b-4abe-b2b1-153dc2405640 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:54:57.810010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dd248-3c6b-4abe-b2b1-153dc2405640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:54:57.810289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dd248-3c6b-4abe-b2b1-153dc2405640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:54:57.812238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-325dd248-3c6b-4abe-b2b1-153dc2405640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:54:57.838864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:54:57.922938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dd248-3c6b-4abe-b2b1-153dc2405640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:54:57.924307 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dd248-3c6b-4abe-b2b1-153dc2405640 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:54:57.942763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 671/1334] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:55:00.043751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc9bd1-d6a6-4692-9edd-7191d6136c41 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:00.044336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc9bd1-d6a6-4692-9edd-7191d6136c41 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:00.044336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc9bd1-d6a6-4692-9edd-7191d6136c41 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:00.045773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 664/1335] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:00.051378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47034614-07b8-4fd9-a7ba-1bfc577d2a27 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:00.051820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47034614-07b8-4fd9-a7ba-1bfc577d2a27 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:00.051939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47034614-07b8-4fd9-a7ba-1bfc577d2a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:00.117565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-47034614-07b8-4fd9-a7ba-1bfc577d2a27 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:00.118803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-47034614-07b8-4fd9-a7ba-1bfc577d2a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:00.194091 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:00.217307 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 672/1336] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:00.258060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc9bd1-d6a6-4692-9edd-7191d6136c41 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:00.259617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f8cc9bd1-d6a6-4692-9edd-7191d6136c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:00.363782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc9bd1-d6a6-4692-9edd-7191d6136c41 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:00.392153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:00.438184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82dbd4-fe50-41c9-8c0e-74b835c84999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:00.438823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82dbd4-fe50-41c9-8c0e-74b835c84999 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:00.439103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82dbd4-fe50-41c9-8c0e-74b835c84999 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:00.439376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82dbd4-fe50-41c9-8c0e-74b835c84999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:00.441024 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fc82dbd4-fe50-41c9-8c0e-74b835c84999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:00.553877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82dbd4-fe50-41c9-8c0e-74b835c84999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:00.555376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82dbd4-fe50-41c9-8c0e-74b835c84999 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:00.574403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 665/1337] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:02.951334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cfce1-6006-4a92-a8f7-23eb9fd4b19b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:02.951911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cfce1-6006-4a92-a8f7-23eb9fd4b19b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:02.951911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cfce1-6006-4a92-a8f7-23eb9fd4b19b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:02.953326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 673/1338] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:02.958742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703ac4b-6c86-4430-b4ee-b008760641ec None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:02.959192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703ac4b-6c86-4430-b4ee-b008760641ec None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:02.959318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703ac4b-6c86-4430-b4ee-b008760641ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:03.037699 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b703ac4b-6c86-4430-b4ee-b008760641ec None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:03.038932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b703ac4b-6c86-4430-b4ee-b008760641ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:03.047758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:03.142378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 666/1339] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:03.183136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cfce1-6006-4a92-a8f7-23eb9fd4b19b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:03.184737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cc5cfce1-6006-4a92-a8f7-23eb9fd4b19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:03.290709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cfce1-6006-4a92-a8f7-23eb9fd4b19b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:03.366184 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1aa33-c64b-40bb-aa0f-77112f4027ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:03.366836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1aa33-c64b-40bb-aa0f-77112f4027ff service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:03.367120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1aa33-c64b-40bb-aa0f-77112f4027ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:03.367401 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1aa33-c64b-40bb-aa0f-77112f4027ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:03.369048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d9b1aa33-c64b-40bb-aa0f-77112f4027ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:03.405601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:03.481135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1aa33-c64b-40bb-aa0f-77112f4027ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:03.482592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1aa33-c64b-40bb-aa0f-77112f4027ff service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:03.505668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 674/1340] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:07.527994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04240a51-1663-433f-a0c7-f9ec7d801b82 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:07.528515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04240a51-1663-433f-a0c7-f9ec7d801b82 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:07.528515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04240a51-1663-433f-a0c7-f9ec7d801b82 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:07.533542 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 667/1341] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:07.536235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181601a-df00-4fe1-a189-d379a5fe1923 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:07.536734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181601a-df00-4fe1-a189-d379a5fe1923 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:07.536872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181601a-df00-4fe1-a189-d379a5fe1923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:07.610903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8181601a-df00-4fe1-a189-d379a5fe1923 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:07.612153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8181601a-df00-4fe1-a189-d379a5fe1923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:07.709279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:07.723830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 675/1342] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:55:07.763224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04240a51-1663-433f-a0c7-f9ec7d801b82 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:07.766027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-04240a51-1663-433f-a0c7-f9ec7d801b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:07.861198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04240a51-1663-433f-a0c7-f9ec7d801b82 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:07.864770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fafb5b-4ac4-41e6-af80-312824a40555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:07.865450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fafb5b-4ac4-41e6-af80-312824a40555 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:07.865739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fafb5b-4ac4-41e6-af80-312824a40555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:07.866029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fafb5b-4ac4-41e6-af80-312824a40555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:07.875077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fafb5b-4ac4-41e6-af80-312824a40555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:07.876507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fafb5b-4ac4-41e6-af80-312824a40555 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:07.886155 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 668/1343] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:10.560869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1bd97-dcfa-45b9-8275-db307b78a64f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:10.561690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1bd97-dcfa-45b9-8275-db307b78a64f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:10.561784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1bd97-dcfa-45b9-8275-db307b78a64f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:10.563622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 676/1344] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:10.568951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979b359-305e-474c-aa24-0b2e97638d4c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:10.569450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979b359-305e-474c-aa24-0b2e97638d4c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:10.569547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979b359-305e-474c-aa24-0b2e97638d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:10.597827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3979b359-305e-474c-aa24-0b2e97638d4c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:10.598956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3979b359-305e-474c-aa24-0b2e97638d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:10.629266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:10.680619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 669/1345] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:10.720792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1bd97-dcfa-45b9-8275-db307b78a64f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:10.731889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1bd97-dcfa-45b9-8275-db307b78a64f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:10.735745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3dc04-9e04-4844-b549-3e7bf866f832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:10.736415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3dc04-9e04-4844-b549-3e7bf866f832 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:10.736697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3dc04-9e04-4844-b549-3e7bf866f832 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:10.736972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3dc04-9e04-4844-b549-3e7bf866f832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:10.746897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3dc04-9e04-4844-b549-3e7bf866f832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:10.748366 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3dc04-9e04-4844-b549-3e7bf866f832 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:10.759563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 677/1346] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:14.801754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca12f2c-24c6-43da-93dc-46528401a3c1 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:14.802323 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca12f2c-24c6-43da-93dc-46528401a3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:14.802323 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca12f2c-24c6-43da-93dc-46528401a3c1 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:14.803738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 670/1347] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:55:14.809028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f096397-5f23-473b-9abb-165fcf9c8cd4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:14.809456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f096397-5f23-473b-9abb-165fcf9c8cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:14.809625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f096397-5f23-473b-9abb-165fcf9c8cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:14.827277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2f096397-5f23-473b-9abb-165fcf9c8cd4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:14.829009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2f096397-5f23-473b-9abb-165fcf9c8cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:14.836894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 678/1348] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:14.875192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca12f2c-24c6-43da-93dc-46528401a3c1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:14.888875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca12f2c-24c6-43da-93dc-46528401a3c1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:14.893197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0945b53-097c-469f-84b1-c5b532d416d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:14.894117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0945b53-097c-469f-84b1-c5b532d416d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:14.894117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0945b53-097c-469f-84b1-c5b532d416d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:14.894400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0945b53-097c-469f-84b1-c5b532d416d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:14.904424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0945b53-097c-469f-84b1-c5b532d416d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:14.905862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0945b53-097c-469f-84b1-c5b532d416d6 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:14.915834 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 671/1349] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:20.026711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e7fc7-171c-4ea9-84ec-f1850af36821 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:20.027561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e7fc7-171c-4ea9-84ec-f1850af36821 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:20.028071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e7fc7-171c-4ea9-84ec-f1850af36821 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:20.030300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 679/1350] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 18:55:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:20.036655 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b34f5c-095d-476e-8e12-687e6568c1fd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:20.037139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b34f5c-095d-476e-8e12-687e6568c1fd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:20.037287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b34f5c-095d-476e-8e12-687e6568c1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:20.127347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a3b34f5c-095d-476e-8e12-687e6568c1fd None None] MFA Rules not processed for user `f0e3fc619dc74a0591d8ad14369ae673`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:20.128898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a3b34f5c-095d-476e-8e12-687e6568c1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:20.244616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 672/1351] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 18:55:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:24.472007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186221e-c465-4e5f-8865-fce84172392e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:24.472643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186221e-c465-4e5f-8865-fce84172392e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:24.472988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186221e-c465-4e5f-8865-fce84172392e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:24.475509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 680/1352] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 18:55:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:24.480457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c81cb5-4b5c-4960-9e70-3982a63c9d70 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:24.480979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c81cb5-4b5c-4960-9e70-3982a63c9d70 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:24.481064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c81cb5-4b5c-4960-9e70-3982a63c9d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:24.500559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-56c81cb5-4b5c-4960-9e70-3982a63c9d70 None None] MFA Rules not processed for user `f0e3fc619dc74a0591d8ad14369ae673`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:24.501874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-56c81cb5-4b5c-4960-9e70-3982a63c9d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:24.510800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 673/1353] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 18:55:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:24.822505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=13 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:26.243637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762718cc-8451-489b-81b4-d1b164e718fd None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:26.244281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762718cc-8451-489b-81b4-d1b164e718fd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:26.244281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762718cc-8451-489b-81b4-d1b164e718fd None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:26.245819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 681/1354] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 18:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:26.251520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b486c-2b88-4f97-b95a-f8090ed8953e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:26.252235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b486c-2b88-4f97-b95a-f8090ed8953e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:26.252235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b486c-2b88-4f97-b95a-f8090ed8953e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:26.274230 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4a0b486c-2b88-4f97-b95a-f8090ed8953e None None] MFA Rules not processed for user `f0e3fc619dc74a0591d8ad14369ae673`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:26.275959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4a0b486c-2b88-4f97-b95a-f8090ed8953e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:26.286433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 674/1355] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 18:55:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:30.036950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6bcca-be36-4bea-b3b1-1b3c1dce6b14 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:30.037856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6bcca-be36-4bea-b3b1-1b3c1dce6b14 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:30.038292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6bcca-be36-4bea-b3b1-1b3c1dce6b14 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:30.039887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 682/1356] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:30.045519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac5940-7d06-487e-8867-e3dbf9a74b1c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:30.045996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac5940-7d06-487e-8867-e3dbf9a74b1c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:30.046094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac5940-7d06-487e-8867-e3dbf9a74b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:30.066355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-54ac5940-7d06-487e-8867-e3dbf9a74b1c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:30.067886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-54ac5940-7d06-487e-8867-e3dbf9a74b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:30.076171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 675/1357] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:30.134125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff93b-5a63-4739-81ec-d416101930ac None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:30.134381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff93b-5a63-4739-81ec-d416101930ac None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:30.134605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff93b-5a63-4739-81ec-d416101930ac None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:30.136139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 683/1358] 10.222.0.70 () {54 vars in 961 bytes} [Tue Jun 21 18:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:30.147183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea8df9-2ef3-4f4e-baf7-25198c304046 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:30.147605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea8df9-2ef3-4f4e-baf7-25198c304046 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:30.148305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea8df9-2ef3-4f4e-baf7-25198c304046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:30.195455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:30.205071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-19ea8df9-2ef3-4f4e-baf7-25198c304046 None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:30.206307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-19ea8df9-2ef3-4f4e-baf7-25198c304046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:30.297473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 676/1359] 10.222.0.70 () {58 vars in 1058 bytes} [Tue Jun 21 18:55:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:30.305669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ff93b-5a63-4739-81ec-d416101930ac None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:30.323424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ff93b-5a63-4739-81ec-d416101930ac None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:30.330611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61ee9f-c809-4d99-86ec-61655a46e7d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:30.331398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61ee9f-c809-4d99-86ec-61655a46e7d9 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:30.331878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61ee9f-c809-4d99-86ec-61655a46e7d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:30.332257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61ee9f-c809-4d99-86ec-61655a46e7d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:30.341524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61ee9f-c809-4d99-86ec-61655a46e7d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:30.343283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61ee9f-c809-4d99-86ec-61655a46e7d9 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:30.352998 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 684/1360] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:32.643118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc56a0-5237-4bfb-a202-578760e504de None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:32.643657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc56a0-5237-4bfb-a202-578760e504de None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:32.643806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc56a0-5237-4bfb-a202-578760e504de None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:32.645250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 677/1361] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:32.650820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e105b2-ee27-4d38-b7c5-8a90ae0092c9 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:32.651517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e105b2-ee27-4d38-b7c5-8a90ae0092c9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:32.651517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e105b2-ee27-4d38-b7c5-8a90ae0092c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:32.671041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-53e105b2-ee27-4d38-b7c5-8a90ae0092c9 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:32.672320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-53e105b2-ee27-4d38-b7c5-8a90ae0092c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:32.681043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 685/1362] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:32.727001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc56a0-5237-4bfb-a202-578760e504de None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:32.743887 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc56a0-5237-4bfb-a202-578760e504de None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:32.750528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4743e-e241-4d4d-8fb4-85cc837b16a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:32.751310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4743e-e241-4d4d-8fb4-85cc837b16a1 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:32.751600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4743e-e241-4d4d-8fb4-85cc837b16a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:32.751883 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4743e-e241-4d4d-8fb4-85cc837b16a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:32.762215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4743e-e241-4d4d-8fb4-85cc837b16a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:32.764040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4743e-e241-4d4d-8fb4-85cc837b16a1 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:32.776757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 678/1363] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:35.155058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5449f3-64f7-406a-b9c3-dd36a5dc6139 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:35.155565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5449f3-64f7-406a-b9c3-dd36a5dc6139 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:35.155565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5449f3-64f7-406a-b9c3-dd36a5dc6139 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:35.157314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 686/1364] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:35.162970 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59755bde-d82c-4989-9a4c-9617a26e2305 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:35.163825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59755bde-d82c-4989-9a4c-9617a26e2305 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:35.164264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59755bde-d82c-4989-9a4c-9617a26e2305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:35.182580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-59755bde-d82c-4989-9a4c-9617a26e2305 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:35.183793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-59755bde-d82c-4989-9a4c-9617a26e2305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:35.192399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 679/1365] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:35.237822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775e81f-59ef-4c13-8d85-04cfebe79272 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:35.238109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775e81f-59ef-4c13-8d85-04cfebe79272 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:35.238387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775e81f-59ef-4c13-8d85-04cfebe79272 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:35.239807 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 687/1366] 10.222.0.70 () {54 vars in 961 bytes} [Tue Jun 21 18:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:35.245194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ee3c-35b0-46a4-bae9-0331bf988bea None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:35.245493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ee3c-35b0-46a4-bae9-0331bf988bea None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:35.245778 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ee3c-35b0-46a4-bae9-0331bf988bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:35.268176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1558ee3c-35b0-46a4-bae9-0331bf988bea None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:35.268176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1558ee3c-35b0-46a4-bae9-0331bf988bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:35.277155 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 680/1367] 10.222.0.70 () {58 vars in 1058 bytes} [Tue Jun 21 18:55:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:35.285309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775e81f-59ef-4c13-8d85-04cfebe79272 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:35.300060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775e81f-59ef-4c13-8d85-04cfebe79272 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:35.305585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5fa5b-09e5-4ee6-a2a3-52201f6f8e54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:35.306231 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5fa5b-09e5-4ee6-a2a3-52201f6f8e54 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:35.306518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5fa5b-09e5-4ee6-a2a3-52201f6f8e54 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:35.306800 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5fa5b-09e5-4ee6-a2a3-52201f6f8e54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:35.315754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5fa5b-09e5-4ee6-a2a3-52201f6f8e54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:35.317360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5fa5b-09e5-4ee6-a2a3-52201f6f8e54 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:35.331283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 688/1368] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:37.430449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0d0dd-36b6-4cce-baf6-ce86afa1ecfc None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:37.430980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0d0dd-36b6-4cce-baf6-ce86afa1ecfc None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:37.431264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0d0dd-36b6-4cce-baf6-ce86afa1ecfc None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:37.432945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 681/1369] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:37.438866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316afb2e-cb55-499d-8afd-5f42caa9e88b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:37.439370 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316afb2e-cb55-499d-8afd-5f42caa9e88b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:37.439454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316afb2e-cb55-499d-8afd-5f42caa9e88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:37.458228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-316afb2e-cb55-499d-8afd-5f42caa9e88b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:37.459444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-316afb2e-cb55-499d-8afd-5f42caa9e88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:37.467381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 689/1370] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:55:37.513314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0d0dd-36b6-4cce-baf6-ce86afa1ecfc None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:37.530610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0d0dd-36b6-4cce-baf6-ce86afa1ecfc None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:37.536703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dfcc3-f301-4237-abd7-3446899426fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:37.537588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dfcc3-f301-4237-abd7-3446899426fb service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:37.538018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dfcc3-f301-4237-abd7-3446899426fb service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:37.538439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dfcc3-f301-4237-abd7-3446899426fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:37.548658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dfcc3-f301-4237-abd7-3446899426fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:37.550752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dfcc3-f301-4237-abd7-3446899426fb service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:37.561170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 682/1371] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:38.224716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c0633-7784-4ada-a03c-2462b5a18b4b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:38.225224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c0633-7784-4ada-a03c-2462b5a18b4b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:38.225547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c0633-7784-4ada-a03c-2462b5a18b4b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:38.226981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 690/1372] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:38.232490 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc843d86-0217-40df-939e-f3b07dd462cb None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:38.233023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc843d86-0217-40df-939e-f3b07dd462cb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:38.233508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc843d86-0217-40df-939e-f3b07dd462cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:38.252783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fc843d86-0217-40df-939e-f3b07dd462cb None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:38.254289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fc843d86-0217-40df-939e-f3b07dd462cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:38.262657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 683/1373] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:38.573907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd82ba5-76a3-4b2b-8022-a2cf926a69b8 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:38.574404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd82ba5-76a3-4b2b-8022-a2cf926a69b8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:38.574541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd82ba5-76a3-4b2b-8022-a2cf926a69b8 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:38.575985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 691/1374] 10.222.0.70 () {54 vars in 963 bytes} [Tue Jun 21 18:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:38.581988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7c1dd-f3c4-4f88-8066-286d969ae19b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:38.582533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7c1dd-f3c4-4f88-8066-286d969ae19b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:38.582711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7c1dd-f3c4-4f88-8066-286d969ae19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:38.642224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-96c7c1dd-f3c4-4f88-8066-286d969ae19b None None] MFA Rules not processed for user `5b8161446e1344108f8283d46da23ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:38.643520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-96c7c1dd-f3c4-4f88-8066-286d969ae19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:38.720237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 684/1375] 10.222.0.70 () {58 vars in 1060 bytes} [Tue Jun 21 18:55:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:38.728233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd82ba5-76a3-4b2b-8022-a2cf926a69b8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:38.739648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd82ba5-76a3-4b2b-8022-a2cf926a69b8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:38.743304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fad0d-a288-4353-9818-da16f4f17ba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:38.744039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fad0d-a288-4353-9818-da16f4f17ba1 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:38.744397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fad0d-a288-4353-9818-da16f4f17ba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:38.744744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fad0d-a288-4353-9818-da16f4f17ba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:38.755613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fad0d-a288-4353-9818-da16f4f17ba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:38.757445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fad0d-a288-4353-9818-da16f4f17ba1 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:38.776144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 692/1376] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:39.705724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949ca0f-cfe2-449e-9ffc-8a7e4049994a None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:39.705724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949ca0f-cfe2-449e-9ffc-8a7e4049994a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:39.705724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949ca0f-cfe2-449e-9ffc-8a7e4049994a None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:39.707295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 685/1377] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:39.714962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc667db3-0b38-4170-acb0-00fccba0c612 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:39.716281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc667db3-0b38-4170-acb0-00fccba0c612 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:39.717122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc667db3-0b38-4170-acb0-00fccba0c612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:39.750954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-dc667db3-0b38-4170-acb0-00fccba0c612 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:39.752644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dc667db3-0b38-4170-acb0-00fccba0c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:39.769850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 693/1378] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:39.819480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949ca0f-cfe2-449e-9ffc-8a7e4049994a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:39.836309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949ca0f-cfe2-449e-9ffc-8a7e4049994a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:39.846878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc679685-fa11-414c-9530-2f34b00ba3df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:39.853331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc679685-fa11-414c-9530-2f34b00ba3df service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:39.853647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc679685-fa11-414c-9530-2f34b00ba3df service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:39.853937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc679685-fa11-414c-9530-2f34b00ba3df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:39.873033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc679685-fa11-414c-9530-2f34b00ba3df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:39.875553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc679685-fa11-414c-9530-2f34b00ba3df service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:39.882521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:39.893361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 686/1379] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:41.992739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9f438-32c7-42b6-a236-92b7583d6f45 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:41.993337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9f438-32c7-42b6-a236-92b7583d6f45 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:41.993337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9f438-32c7-42b6-a236-92b7583d6f45 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:41.994682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 694/1380] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:42.000073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a002e44-352d-4bb0-8ca3-94adf720886d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:42.000492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a002e44-352d-4bb0-8ca3-94adf720886d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:42.000643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a002e44-352d-4bb0-8ca3-94adf720886d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:42.018553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5a002e44-352d-4bb0-8ca3-94adf720886d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:42.019682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5a002e44-352d-4bb0-8ca3-94adf720886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:42.027910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 687/1381] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:42.072915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9f438-32c7-42b6-a236-92b7583d6f45 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:42.087104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9f438-32c7-42b6-a236-92b7583d6f45 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:42.092422 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a558c-08f7-4017-aa50-8b1ba2900d83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:42.093674 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a558c-08f7-4017-aa50-8b1ba2900d83 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:42.093962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a558c-08f7-4017-aa50-8b1ba2900d83 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:42.094239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a558c-08f7-4017-aa50-8b1ba2900d83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:42.102542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a558c-08f7-4017-aa50-8b1ba2900d83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:42.104122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a558c-08f7-4017-aa50-8b1ba2900d83 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:42.115039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 695/1382] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:44.205440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21341d1f-0e50-42b4-9959-f58842bb1588 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:44.206398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21341d1f-0e50-42b4-9959-f58842bb1588 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:44.206931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21341d1f-0e50-42b4-9959-f58842bb1588 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:44.208835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 688/1383] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:44.213970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172af85-7e13-4c5c-bce9-3ad9952fc99b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:44.214474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172af85-7e13-4c5c-bce9-3ad9952fc99b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:44.214579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172af85-7e13-4c5c-bce9-3ad9952fc99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:44.232772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0172af85-7e13-4c5c-bce9-3ad9952fc99b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:44.234031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0172af85-7e13-4c5c-bce9-3ad9952fc99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:44.242051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 696/1384] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:44.284532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21341d1f-0e50-42b4-9959-f58842bb1588 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:44.299465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21341d1f-0e50-42b4-9959-f58842bb1588 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:44.305342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7e780-a5fb-44a4-87cb-e51eeaa2ca2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:44.306077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7e780-a5fb-44a4-87cb-e51eeaa2ca2e service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:44.306456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7e780-a5fb-44a4-87cb-e51eeaa2ca2e service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:44.306811 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7e780-a5fb-44a4-87cb-e51eeaa2ca2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:44.317687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7e780-a5fb-44a4-87cb-e51eeaa2ca2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:44.319427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7e780-a5fb-44a4-87cb-e51eeaa2ca2e service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:44.331596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 689/1385] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:46.416876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9885f-7eda-4ef3-8985-1cd5a7fd9c78 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:46.417327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9885f-7eda-4ef3-8985-1cd5a7fd9c78 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:46.417573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9885f-7eda-4ef3-8985-1cd5a7fd9c78 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:46.419059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 697/1386] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:46.424930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a6c43-8ff1-4acf-ba8d-ba5c3846d87d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:46.425415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a6c43-8ff1-4acf-ba8d-ba5c3846d87d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:46.425415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a6c43-8ff1-4acf-ba8d-ba5c3846d87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:46.446265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-718a6c43-8ff1-4acf-ba8d-ba5c3846d87d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:46.448103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-718a6c43-8ff1-4acf-ba8d-ba5c3846d87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:46.462578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 690/1387] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:46.502141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9885f-7eda-4ef3-8985-1cd5a7fd9c78 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:46.517998 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9885f-7eda-4ef3-8985-1cd5a7fd9c78 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:46.523565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19addcba-a4f8-4980-a7be-ea67b676a630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:46.524230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19addcba-a4f8-4980-a7be-ea67b676a630 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:46.524518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19addcba-a4f8-4980-a7be-ea67b676a630 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:46.524802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19addcba-a4f8-4980-a7be-ea67b676a630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:46.534808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19addcba-a4f8-4980-a7be-ea67b676a630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:46.536429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19addcba-a4f8-4980-a7be-ea67b676a630 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:46.547459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 698/1388] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:48.673477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cadc7-f95c-4801-8c5c-3fc773d05573 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:48.674131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cadc7-f95c-4801-8c5c-3fc773d05573 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:48.674131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cadc7-f95c-4801-8c5c-3fc773d05573 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:48.675346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 691/1389] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:48.681281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3d0f-4284-420f-8b5d-49ff1a6fa9d5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:48.681797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3d0f-4284-420f-8b5d-49ff1a6fa9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:48.681972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3d0f-4284-420f-8b5d-49ff1a6fa9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:48.705230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-774d3d0f-4284-420f-8b5d-49ff1a6fa9d5 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:48.706877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-774d3d0f-4284-420f-8b5d-49ff1a6fa9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:48.715857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 699/1390] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:48.761063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cadc7-f95c-4801-8c5c-3fc773d05573 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:48.776469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cadc7-f95c-4801-8c5c-3fc773d05573 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:48.783418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd5d46-f814-46a3-ad06-b3aa64c59d65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:48.784263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd5d46-f814-46a3-ad06-b3aa64c59d65 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:48.784621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd5d46-f814-46a3-ad06-b3aa64c59d65 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:48.784972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd5d46-f814-46a3-ad06-b3aa64c59d65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:48.795978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd5d46-f814-46a3-ad06-b3aa64c59d65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:48.798009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd5d46-f814-46a3-ad06-b3aa64c59d65 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:48.810925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 692/1391] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:51.136682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f789ac-21a1-49d6-b408-e986b5008fde None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:51.137376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f789ac-21a1-49d6-b408-e986b5008fde None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:51.137376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f789ac-21a1-49d6-b408-e986b5008fde None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:51.138657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 700/1392] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:51.144822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a3bf6-29dd-43a8-b7de-34802c37e409 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:51.145335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a3bf6-29dd-43a8-b7de-34802c37e409 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:51.145615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a3bf6-29dd-43a8-b7de-34802c37e409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:51.167957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-867a3bf6-29dd-43a8-b7de-34802c37e409 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:51.169205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-867a3bf6-29dd-43a8-b7de-34802c37e409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:51.177776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 693/1393] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:51.221833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f789ac-21a1-49d6-b408-e986b5008fde None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:51.237484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f789ac-21a1-49d6-b408-e986b5008fde None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:51.243246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bfbea-7cf9-4a8b-a235-07f04b1d8db4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:51.243903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bfbea-7cf9-4a8b-a235-07f04b1d8db4 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:51.244195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bfbea-7cf9-4a8b-a235-07f04b1d8db4 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:51.244482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bfbea-7cf9-4a8b-a235-07f04b1d8db4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:51.253867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bfbea-7cf9-4a8b-a235-07f04b1d8db4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:51.255469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bfbea-7cf9-4a8b-a235-07f04b1d8db4 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:51.265891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 701/1394] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:53.719282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171b1a-e914-4f3f-abb8-7839ebd230b2 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:53.719886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171b1a-e914-4f3f-abb8-7839ebd230b2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:53.719886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5171b1a-e914-4f3f-abb8-7839ebd230b2 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:53.721232 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 694/1395] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:53.726707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ad00d-7d7c-4d7a-8256-97b552ce83c8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:53.727210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ad00d-7d7c-4d7a-8256-97b552ce83c8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:53.727311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ad00d-7d7c-4d7a-8256-97b552ce83c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:53.751742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-aa3ad00d-7d7c-4d7a-8256-97b552ce83c8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:53.752725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aa3ad00d-7d7c-4d7a-8256-97b552ce83c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:53.761528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 702/1396] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:53.805834 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171b1a-e914-4f3f-abb8-7839ebd230b2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:53.821466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5171b1a-e914-4f3f-abb8-7839ebd230b2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:53.827007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dd5b0-8cc3-45be-80bf-ce52737e6efc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:53.827700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dd5b0-8cc3-45be-80bf-ce52737e6efc service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:53.827982 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dd5b0-8cc3-45be-80bf-ce52737e6efc service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:53.828260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dd5b0-8cc3-45be-80bf-ce52737e6efc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:53.836996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dd5b0-8cc3-45be-80bf-ce52737e6efc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:53.838682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dd5b0-8cc3-45be-80bf-ce52737e6efc service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:53.848274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 695/1397] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:55.253078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5ce27-eb8b-44c5-b42d-21254987ed0f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:55.253675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5ce27-eb8b-44c5-b42d-21254987ed0f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:55.253675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5ce27-eb8b-44c5-b42d-21254987ed0f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:55.255086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 703/1398] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:55.260433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542dae4-5359-4f55-a9d4-24e41bd795fe None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:55.260866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542dae4-5359-4f55-a9d4-24e41bd795fe None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:55.260998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542dae4-5359-4f55-a9d4-24e41bd795fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:55.279631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a542dae4-5359-4f55-a9d4-24e41bd795fe None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:55.280884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a542dae4-5359-4f55-a9d4-24e41bd795fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:55.289055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 696/1399] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:55.347547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebd7fd-f4c5-48c4-850a-931b3e7ef14f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:55.347839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebd7fd-f4c5-48c4-850a-931b3e7ef14f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:55.348136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebd7fd-f4c5-48c4-850a-931b3e7ef14f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:55.349565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 704/1400] 10.222.0.70 () {54 vars in 962 bytes} [Tue Jun 21 18:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:55.355402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fa3e2-46fe-4c18-9bc5-b7310b852624 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:55.355694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fa3e2-46fe-4c18-9bc5-b7310b852624 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:55.355976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fa3e2-46fe-4c18-9bc5-b7310b852624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:55.410963 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d44fa3e2-46fe-4c18-9bc5-b7310b852624 None None] MFA Rules not processed for user `915704f26c814eaa9e6c2a68533a9c66`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:55.412253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d44fa3e2-46fe-4c18-9bc5-b7310b852624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:55.479848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 697/1401] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Jun 21 18:55:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:55.488413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ebd7fd-f4c5-48c4-850a-931b3e7ef14f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:55.496462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=55 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:55:55.502297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ebd7fd-f4c5-48c4-850a-931b3e7ef14f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:55.506028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2be08-c0a3-4a62-bded-b73549034406 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915704f26c814eaa9e6c2a68533a9c66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:55.506641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2be08-c0a3-4a62-bded-b73549034406 service glance] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:55.506964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2be08-c0a3-4a62-bded-b73549034406 service glance] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:55.507262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2be08-c0a3-4a62-bded-b73549034406 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:55.517581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2be08-c0a3-4a62-bded-b73549034406 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:55.519017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2be08-c0a3-4a62-bded-b73549034406 service glance] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:55.529103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 705/1402] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:55:55.938211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eed7be-5c07-4071-93a4-796afaed7416 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:55.938491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eed7be-5c07-4071-93a4-796afaed7416 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:55.938777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eed7be-5c07-4071-93a4-796afaed7416 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:55.940420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 698/1403] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:55:55.946007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2708b-c487-4bc2-9ad8-bfdac9896abc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:55.946299 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2708b-c487-4bc2-9ad8-bfdac9896abc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:55.946599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2708b-c487-4bc2-9ad8-bfdac9896abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:55.965359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6de2708b-c487-4bc2-9ad8-bfdac9896abc None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:55.966533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6de2708b-c487-4bc2-9ad8-bfdac9896abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:55.974877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 706/1404] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:56.019047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eed7be-5c07-4071-93a4-796afaed7416 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:56.034202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eed7be-5c07-4071-93a4-796afaed7416 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:56.039933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02193fed-cd7a-4e5f-a9a5-680f521b0586 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:56.040579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02193fed-cd7a-4e5f-a9a5-680f521b0586 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:56.040867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02193fed-cd7a-4e5f-a9a5-680f521b0586 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:56.041184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02193fed-cd7a-4e5f-a9a5-680f521b0586 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:56.049877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02193fed-cd7a-4e5f-a9a5-680f521b0586 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:56.051468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02193fed-cd7a-4e5f-a9a5-680f521b0586 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:56.062790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 699/1405] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 18:55:56.260352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709afe0-0209-41c9-8858-b1751f6a52df None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:56.260895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709afe0-0209-41c9-8858-b1751f6a52df None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:56.260985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709afe0-0209-41c9-8858-b1751f6a52df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:56.313521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6709afe0-0209-41c9-8858-b1751f6a52df None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:56.314729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6709afe0-0209-41c9-8858-b1751f6a52df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:56.379756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 707/1406] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 18:55:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:56.424765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91935247-5144-47d5-beb3-971533814fe1 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:56.425311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91935247-5144-47d5-beb3-971533814fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:56.425420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91935247-5144-47d5-beb3-971533814fe1 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:56.426757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 700/1407] 10.222.0.70 () {54 vars in 942 bytes} [Tue Jun 21 18:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:55:56.433406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7827cd0-30a0-4280-b7dd-f8cda06e3d8b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:56.433640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7827cd0-30a0-4280-b7dd-f8cda06e3d8b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:56.433923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7827cd0-30a0-4280-b7dd-f8cda06e3d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:56.485376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a7827cd0-30a0-4280-b7dd-f8cda06e3d8b None None] MFA Rules not processed for user `ad2a56d3235b46e3b1fbb8bf0c2b4e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:55:56.486628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a7827cd0-30a0-4280-b7dd-f8cda06e3d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:55:56.560078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 708/1408] 10.222.0.70 () {58 vars in 1039 bytes} [Tue Jun 21 18:55:56 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:55:56.570180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91935247-5144-47d5-beb3-971533814fe1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:55:56.583691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91935247-5144-47d5-beb3-971533814fe1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:55:56.588645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf91b4-afd1-453c-85db-883c262de209 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:55:56.589344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf91b4-afd1-453c-85db-883c262de209 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:55:56.589636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf91b4-afd1-453c-85db-883c262de209 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:55:56.589921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf91b4-afd1-453c-85db-883c262de209 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:55:56.598925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf91b4-afd1-453c-85db-883c262de209 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:55:56.600807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf91b4-afd1-453c-85db-883c262de209 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:55:56.611291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 701/1409] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 18:55:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 18:56:06.557129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:56:06.607277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=81 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:56:18.669854 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ca666-a566-4ec3-909d-122d3eaead0c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:18.670382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ca666-a566-4ec3-909d-122d3eaead0c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:18.670382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ca666-a566-4ec3-909d-122d3eaead0c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:18.671745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 709/1410] 10.222.0.70 () {54 vars in 918 bytes} [Tue Jun 21 18:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:18.676925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32802d-cf3e-42ab-8e44-69c0187a0c53 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:18.677335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32802d-cf3e-42ab-8e44-69c0187a0c53 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:18.677526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32802d-cf3e-42ab-8e44-69c0187a0c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:18.695921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6a32802d-cf3e-42ab-8e44-69c0187a0c53 None None] MFA Rules not processed for user `f0e3fc619dc74a0591d8ad14369ae673`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:18.697152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6a32802d-cf3e-42ab-8e44-69c0187a0c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:18.706264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 702/1411] 10.222.0.70 () {58 vars in 1015 bytes} [Tue Jun 21 18:56:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:18.720247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cdd74-6213-4312-84d1-5032672ae20e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:18.720549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cdd74-6213-4312-84d1-5032672ae20e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:18.720833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cdd74-6213-4312-84d1-5032672ae20e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:18.722213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 710/1412] 10.222.0.70 () {54 vars in 918 bytes} [Tue Jun 21 18:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:18.726590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75716d-10ca-4c5a-9394-ff715493c2f2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:18.726885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75716d-10ca-4c5a-9394-ff715493c2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:18.727168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75716d-10ca-4c5a-9394-ff715493c2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:18.744458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ba75716d-10ca-4c5a-9394-ff715493c2f2 None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:18.745676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ba75716d-10ca-4c5a-9394-ff715493c2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:18.755111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 703/1413] 10.222.0.70 () {58 vars in 1015 bytes} [Tue Jun 21 18:56:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:56:21.226750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7e581-de25-4efb-bc02-5dca5e354352 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:21.227371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7e581-de25-4efb-bc02-5dca5e354352 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:21.227371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7e581-de25-4efb-bc02-5dca5e354352 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:21.228775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:21.235328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b022eb-f9c6-43fe-a797-a0d1e655198c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:21.235879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b022eb-f9c6-43fe-a797-a0d1e655198c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:21.235879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b022eb-f9c6-43fe-a797-a0d1e655198c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:21.254349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-78b022eb-f9c6-43fe-a797-a0d1e655198c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:21.255578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-78b022eb-f9c6-43fe-a797-a0d1e655198c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:21.263794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:21.307267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7e581-de25-4efb-bc02-5dca5e354352 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:21.323065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7e581-de25-4efb-bc02-5dca5e354352 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:21.329006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d75fb-3abd-44bb-9972-d3019b2ff89e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:21.329686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d75fb-3abd-44bb-9972-d3019b2ff89e service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:21.329973 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d75fb-3abd-44bb-9972-d3019b2ff89e service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:21.330259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d75fb-3abd-44bb-9972-d3019b2ff89e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:21.339649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d75fb-3abd-44bb-9972-d3019b2ff89e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:21.341291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d75fb-3abd-44bb-9972-d3019b2ff89e service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:21.351396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 712/1416] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:22.255301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c90d9f-319e-40f7-b15f-f4f4a8838b57 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:22.255840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c90d9f-319e-40f7-b15f-f4f4a8838b57 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:22.255957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c90d9f-319e-40f7-b15f-f4f4a8838b57 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:22.257298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:22.262296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b832193-ae55-4682-b026-56ce6edcfc4e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:22.262756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b832193-ae55-4682-b026-56ce6edcfc4e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:22.262850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b832193-ae55-4682-b026-56ce6edcfc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:22.280633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7b832193-ae55-4682-b026-56ce6edcfc4e None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:22.281895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7b832193-ae55-4682-b026-56ce6edcfc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:22.289730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:22.441671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c90d9f-319e-40f7-b15f-f4f4a8838b57 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:22.455992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c90d9f-319e-40f7-b15f-f4f4a8838b57 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:22.461368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1db27-58e5-40fd-b731-d21ea91f444a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:22.462006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1db27-58e5-40fd-b731-d21ea91f444a service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:22.462296 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1db27-58e5-40fd-b731-d21ea91f444a service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:22.462577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1db27-58e5-40fd-b731-d21ea91f444a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:22.471190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1db27-58e5-40fd-b731-d21ea91f444a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:22.472788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1db27-58e5-40fd-b731-d21ea91f444a service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:22.483873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 706/1419] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:23.365006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcada7-a25d-465f-be52-ae79d44fac79 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:23.365575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcada7-a25d-465f-be52-ae79d44fac79 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:23.365673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcada7-a25d-465f-be52-ae79d44fac79 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:23.366979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:23.373263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35914d52-6dd2-41c6-b6fc-ec29932bc843 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:23.374297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35914d52-6dd2-41c6-b6fc-ec29932bc843 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:23.374401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35914d52-6dd2-41c6-b6fc-ec29932bc843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:23.394828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-35914d52-6dd2-41c6-b6fc-ec29932bc843 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:23.396048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-35914d52-6dd2-41c6-b6fc-ec29932bc843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:23.403873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:23.553820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcada7-a25d-465f-be52-ae79d44fac79 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:23.569108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcada7-a25d-465f-be52-ae79d44fac79 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:23.574557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be84f6f-93cd-4513-93e0-45a812a37039 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:23.575134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be84f6f-93cd-4513-93e0-45a812a37039 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:23.575419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be84f6f-93cd-4513-93e0-45a812a37039 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:23.575706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be84f6f-93cd-4513-93e0-45a812a37039 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:23.583931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be84f6f-93cd-4513-93e0-45a812a37039 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:23.585541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be84f6f-93cd-4513-93e0-45a812a37039 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:23.594831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 715/1422] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:24.450806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670af9f6-949b-4dad-9883-3bc0f3772e8a None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:24.451374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670af9f6-949b-4dad-9883-3bc0f3772e8a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:24.451647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670af9f6-949b-4dad-9883-3bc0f3772e8a None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:24.453589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 18:56:24.458498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86c50c-876b-4270-8266-90eefa81075f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:24.458931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86c50c-876b-4270-8266-90eefa81075f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:24.459056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86c50c-876b-4270-8266-90eefa81075f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:24.477337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-be86c50c-876b-4270-8266-90eefa81075f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:24.478535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-be86c50c-876b-4270-8266-90eefa81075f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:24.486334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:24.638299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670af9f6-949b-4dad-9883-3bc0f3772e8a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:24.652791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670af9f6-949b-4dad-9883-3bc0f3772e8a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:24.659254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b172d-6197-459b-aaf8-d56c8c43382d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:24.659907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b172d-6197-459b-aaf8-d56c8c43382d service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:24.660192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b172d-6197-459b-aaf8-d56c8c43382d service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:24.660470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b172d-6197-459b-aaf8-d56c8c43382d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:24.669406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b172d-6197-459b-aaf8-d56c8c43382d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:24.671574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b172d-6197-459b-aaf8-d56c8c43382d service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:24.681080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 709/1425] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:25.550959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fcc05-9dc7-47f1-9248-0a785ecb41f9 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:25.551510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fcc05-9dc7-47f1-9248-0a785ecb41f9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:25.551510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fcc05-9dc7-47f1-9248-0a785ecb41f9 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:25.552877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:25.557927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75b7b-667d-4111-9bc6-5d1a30903baa None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:25.558359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75b7b-667d-4111-9bc6-5d1a30903baa None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:25.558495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75b7b-667d-4111-9bc6-5d1a30903baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:25.576722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-50d75b7b-667d-4111-9bc6-5d1a30903baa None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:25.577886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-50d75b7b-667d-4111-9bc6-5d1a30903baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:25.585631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:25.736928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fcc05-9dc7-47f1-9248-0a785ecb41f9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:25.751015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fcc05-9dc7-47f1-9248-0a785ecb41f9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:25.756224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce14bd5-c204-4a4b-9df3-b1af6418995e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:25.756847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14bd5-c204-4a4b-9df3-b1af6418995e service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:25.757160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14bd5-c204-4a4b-9df3-b1af6418995e service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:25.757439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14bd5-c204-4a4b-9df3-b1af6418995e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:25.767063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce14bd5-c204-4a4b-9df3-b1af6418995e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:25.769947 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce14bd5-c204-4a4b-9df3-b1af6418995e service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:25.780450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 718/1428] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:26.597878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917f176-6a08-4307-b3bc-782bad67041c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:26.598431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917f176-6a08-4307-b3bc-782bad67041c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:26.598431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917f176-6a08-4307-b3bc-782bad67041c None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:26.599792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:26.604910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc16c3-f6d7-48b9-9605-9eca610ad816 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:26.605384 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc16c3-f6d7-48b9-9605-9eca610ad816 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:26.605632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc16c3-f6d7-48b9-9605-9eca610ad816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:26.624449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-befc16c3-f6d7-48b9-9605-9eca610ad816 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:26.625683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-befc16c3-f6d7-48b9-9605-9eca610ad816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:26.633543 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:26.785439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917f176-6a08-4307-b3bc-782bad67041c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:26.800097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917f176-6a08-4307-b3bc-782bad67041c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:26.805458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd4268-4194-460f-a068-e7c5a7ea710e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:26.806095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd4268-4194-460f-a068-e7c5a7ea710e service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:26.806381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd4268-4194-460f-a068-e7c5a7ea710e service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:26.806657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd4268-4194-460f-a068-e7c5a7ea710e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:26.815804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd4268-4194-460f-a068-e7c5a7ea710e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:26.817438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd4268-4194-460f-a068-e7c5a7ea710e service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:26.827608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 712/1431] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:27.632203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673857b-e222-475f-8635-0c63b2cb286e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:27.632725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673857b-e222-475f-8635-0c63b2cb286e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:27.632725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673857b-e222-475f-8635-0c63b2cb286e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:27.634138 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:27.639778 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48071f1a-924f-46d1-a883-beafbd5c896d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:27.640247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48071f1a-924f-46d1-a883-beafbd5c896d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:27.640360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48071f1a-924f-46d1-a883-beafbd5c896d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:27.658812 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-48071f1a-924f-46d1-a883-beafbd5c896d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:27.660029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-48071f1a-924f-46d1-a883-beafbd5c896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:27.667908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:27.819879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673857b-e222-475f-8635-0c63b2cb286e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:27.835080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673857b-e222-475f-8635-0c63b2cb286e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:27.840442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13a201-06f9-4ed4-b1db-ebb7be899b35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:27.841108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13a201-06f9-4ed4-b1db-ebb7be899b35 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:27.841395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13a201-06f9-4ed4-b1db-ebb7be899b35 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:27.841673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13a201-06f9-4ed4-b1db-ebb7be899b35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:27.850396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13a201-06f9-4ed4-b1db-ebb7be899b35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:27.851984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13a201-06f9-4ed4-b1db-ebb7be899b35 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:27.861672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 721/1434] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:28.681367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227e446-d60d-4e1d-b55b-73be6b1aface None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:28.681893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227e446-d60d-4e1d-b55b-73be6b1aface None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:28.681893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227e446-d60d-4e1d-b55b-73be6b1aface None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:28.683322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:28.688758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140115c1-6a9b-4e55-b515-83acf8577225 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:28.689239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140115c1-6a9b-4e55-b515-83acf8577225 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:28.689363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140115c1-6a9b-4e55-b515-83acf8577225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:28.707250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-140115c1-6a9b-4e55-b515-83acf8577225 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:28.708534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-140115c1-6a9b-4e55-b515-83acf8577225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:28.716686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:28.867803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227e446-d60d-4e1d-b55b-73be6b1aface None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:28.882511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227e446-d60d-4e1d-b55b-73be6b1aface None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:28.887975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd8a03-4758-4b58-9363-c1f904ec8939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:28.888636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd8a03-4758-4b58-9363-c1f904ec8939 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:28.888927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd8a03-4758-4b58-9363-c1f904ec8939 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:28.889848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd8a03-4758-4b58-9363-c1f904ec8939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:28.899023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd8a03-4758-4b58-9363-c1f904ec8939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:28.900647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd8a03-4758-4b58-9363-c1f904ec8939 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:28.910826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 715/1437] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:29.716467 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3adb95-6cd1-42c4-8346-eabce5146179 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:29.717028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3adb95-6cd1-42c4-8346-eabce5146179 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:29.717028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3adb95-6cd1-42c4-8346-eabce5146179 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:29.719007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 18:56:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:29.723987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b1f0e-7896-4703-ae8e-2248d96f11d7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:29.724423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b1f0e-7896-4703-ae8e-2248d96f11d7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:29.724556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b1f0e-7896-4703-ae8e-2248d96f11d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:29.743096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9e9b1f0e-7896-4703-ae8e-2248d96f11d7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:29.744305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9e9b1f0e-7896-4703-ae8e-2248d96f11d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:29.751979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 18:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:29.905300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3adb95-6cd1-42c4-8346-eabce5146179 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:29.919581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3adb95-6cd1-42c4-8346-eabce5146179 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:29.925110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f617-1ddd-404e-9626-817e14921c9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:29.925768 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f617-1ddd-404e-9626-817e14921c9e service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:29.926087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f617-1ddd-404e-9626-817e14921c9e service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:29.926365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f617-1ddd-404e-9626-817e14921c9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:29.934725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f617-1ddd-404e-9626-817e14921c9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:29.936815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f617-1ddd-404e-9626-817e14921c9e service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:29.946498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 724/1440] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:31.880331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091725af-670b-4cd5-9dc8-a8568bcb3b38 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:31.880915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091725af-670b-4cd5-9dc8-a8568bcb3b38 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:31.880915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091725af-670b-4cd5-9dc8-a8568bcb3b38 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:31.882329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:31.887722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cceb00-7360-4b1f-a4e1-14e2ba1912c7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:31.888181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cceb00-7360-4b1f-a4e1-14e2ba1912c7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:31.888290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cceb00-7360-4b1f-a4e1-14e2ba1912c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:31.905824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-01cceb00-7360-4b1f-a4e1-14e2ba1912c7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:31.907048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-01cceb00-7360-4b1f-a4e1-14e2ba1912c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:31.914796 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:31.953297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091725af-670b-4cd5-9dc8-a8568bcb3b38 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:31.963989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091725af-670b-4cd5-9dc8-a8568bcb3b38 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:31.967564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef55138-5b7f-4c3c-9d5d-143dceb38fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:31.968202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef55138-5b7f-4c3c-9d5d-143dceb38fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:31.968488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef55138-5b7f-4c3c-9d5d-143dceb38fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:31.968772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef55138-5b7f-4c3c-9d5d-143dceb38fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:31.977764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef55138-5b7f-4c3c-9d5d-143dceb38fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:31.979195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef55138-5b7f-4c3c-9d5d-143dceb38fd1 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:31.989208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 718/1443] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:34.193707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c89d84-5d28-4f17-9689-fe028e436015 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:34.194328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c89d84-5d28-4f17-9689-fe028e436015 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:34.194328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c89d84-5d28-4f17-9689-fe028e436015 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:34.195636 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:34.200949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4861b6c-9023-466e-a9a8-c7f432d00514 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:34.201448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4861b6c-9023-466e-a9a8-c7f432d00514 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:34.202077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4861b6c-9023-466e-a9a8-c7f432d00514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:34.220459 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c4861b6c-9023-466e-a9a8-c7f432d00514 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:34.221708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c4861b6c-9023-466e-a9a8-c7f432d00514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:34.229672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:34.267912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c89d84-5d28-4f17-9689-fe028e436015 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:34.278721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c89d84-5d28-4f17-9689-fe028e436015 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:34.282881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4fd5-1af4-407c-86ee-b9d1ee8a73e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:34.283534 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4fd5-1af4-407c-86ee-b9d1ee8a73e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:34.283822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4fd5-1af4-407c-86ee-b9d1ee8a73e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:34.284103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4fd5-1af4-407c-86ee-b9d1ee8a73e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:34.293294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c4fd5-1af4-407c-86ee-b9d1ee8a73e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:34.294748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c4fd5-1af4-407c-86ee-b9d1ee8a73e0 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:34.304434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 727/1446] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:36.699976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b5b7e-8d6b-4944-9285-33d4bc034406 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:36.700546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b5b7e-8d6b-4944-9285-33d4bc034406 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:36.700546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b5b7e-8d6b-4944-9285-33d4bc034406 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:36.702014 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:36.707677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8e1c3-7984-4f55-bf10-6ea5e77b7e77 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:36.708106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8e1c3-7984-4f55-bf10-6ea5e77b7e77 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:36.708277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8e1c3-7984-4f55-bf10-6ea5e77b7e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:36.727130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-cfc8e1c3-7984-4f55-bf10-6ea5e77b7e77 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:36.728328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cfc8e1c3-7984-4f55-bf10-6ea5e77b7e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:36.736587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:36.775575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b5b7e-8d6b-4944-9285-33d4bc034406 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:36.777389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b80b5b7e-8d6b-4944-9285-33d4bc034406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:36.791157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b5b7e-8d6b-4944-9285-33d4bc034406 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:36.794904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ea5e6-2bb7-4670-b8c7-31b1d4b304f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:36.796202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ea5e6-2bb7-4670-b8c7-31b1d4b304f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:36.796488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ea5e6-2bb7-4670-b8c7-31b1d4b304f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:36.796772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ea5e6-2bb7-4670-b8c7-31b1d4b304f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:36.807126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ea5e6-2bb7-4670-b8c7-31b1d4b304f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:36.808611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ea5e6-2bb7-4670-b8c7-31b1d4b304f9 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:36.820069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 721/1449] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:39.053264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4d8c7-c3a3-4a73-9acd-aff01f57213d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:39.053805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4d8c7-c3a3-4a73-9acd-aff01f57213d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:39.053903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4d8c7-c3a3-4a73-9acd-aff01f57213d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:39.055215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:39.060521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38abf1d-7fd9-4761-a231-3ac7cd78dbfd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:39.061550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38abf1d-7fd9-4761-a231-3ac7cd78dbfd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:39.061784 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38abf1d-7fd9-4761-a231-3ac7cd78dbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:39.080453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d38abf1d-7fd9-4761-a231-3ac7cd78dbfd None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:39.081699 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d38abf1d-7fd9-4761-a231-3ac7cd78dbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:39.089537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:39.128073 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4d8c7-c3a3-4a73-9acd-aff01f57213d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:39.140083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4d8c7-c3a3-4a73-9acd-aff01f57213d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:39.143611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7e2cd-975c-450a-a2b6-cd48c48972a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:39.144263 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7e2cd-975c-450a-a2b6-cd48c48972a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:39.144550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7e2cd-975c-450a-a2b6-cd48c48972a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:39.144831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7e2cd-975c-450a-a2b6-cd48c48972a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:39.154292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7e2cd-975c-450a-a2b6-cd48c48972a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:39.155742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7e2cd-975c-450a-a2b6-cd48c48972a0 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:39.165267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 730/1452] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:41.266698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0ff31-1fd1-4f23-ba20-40856c91a00b None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:41.267273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0ff31-1fd1-4f23-ba20-40856c91a00b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:41.267273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0ff31-1fd1-4f23-ba20-40856c91a00b None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:41.268681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:41.274048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa199b-6847-42c3-a177-b602bf7bc56a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:41.274504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa199b-6847-42c3-a177-b602bf7bc56a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:41.274601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa199b-6847-42c3-a177-b602bf7bc56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:41.292627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-36fa199b-6847-42c3-a177-b602bf7bc56a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:41.293868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-36fa199b-6847-42c3-a177-b602bf7bc56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:41.302264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:56:41.340216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0ff31-1fd1-4f23-ba20-40856c91a00b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:41.351419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0ff31-1fd1-4f23-ba20-40856c91a00b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:41.354950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9827f3-e20c-44a1-bd31-0fd53f512e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:41.355590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9827f3-e20c-44a1-bd31-0fd53f512e0f service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:41.355876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9827f3-e20c-44a1-bd31-0fd53f512e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:41.356150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9827f3-e20c-44a1-bd31-0fd53f512e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:41.366077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9827f3-e20c-44a1-bd31-0fd53f512e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:41.367512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9827f3-e20c-44a1-bd31-0fd53f512e0f service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:41.377306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 724/1455] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:43.563975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f21e6-1b22-4633-86c7-c680d7989e70 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:43.564505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f21e6-1b22-4633-86c7-c680d7989e70 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:43.564505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f21e6-1b22-4633-86c7-c680d7989e70 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:43.565862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:43.571235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb944d-f09e-4275-887f-d44b1c297d4c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:43.571694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb944d-f09e-4275-887f-d44b1c297d4c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:43.571829 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb944d-f09e-4275-887f-d44b1c297d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:43.590673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e1fb944d-f09e-4275-887f-d44b1c297d4c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:43.591872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e1fb944d-f09e-4275-887f-d44b1c297d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:43.600138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:43.638141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f21e6-1b22-4633-86c7-c680d7989e70 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:43.648766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f21e6-1b22-4633-86c7-c680d7989e70 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:43.652288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53f892-a194-4bff-b6a3-7a6727310932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:43.652922 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53f892-a194-4bff-b6a3-7a6727310932 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:43.653236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53f892-a194-4bff-b6a3-7a6727310932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:43.653512 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53f892-a194-4bff-b6a3-7a6727310932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:43.662925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53f892-a194-4bff-b6a3-7a6727310932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:43.664353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53f892-a194-4bff-b6a3-7a6727310932 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:43.675140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 733/1458] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:45.800302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e070b3-3a97-44a2-bb75-295c0d380e93 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:45.800891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e070b3-3a97-44a2-bb75-295c0d380e93 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:45.800891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e070b3-3a97-44a2-bb75-295c0d380e93 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:45.802291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:45.807564 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2500765-2c9d-42d6-b3ce-2f02f3ecaaed None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:45.808012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2500765-2c9d-42d6-b3ce-2f02f3ecaaed None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:45.808122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2500765-2c9d-42d6-b3ce-2f02f3ecaaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:45.826021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c2500765-2c9d-42d6-b3ce-2f02f3ecaaed None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:45.827253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c2500765-2c9d-42d6-b3ce-2f02f3ecaaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:45.835049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:45.873599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e070b3-3a97-44a2-bb75-295c0d380e93 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:45.884405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e070b3-3a97-44a2-bb75-295c0d380e93 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:45.887955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d359ac-c127-4c64-9aac-7ede8e9e6b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:45.888609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d359ac-c127-4c64-9aac-7ede8e9e6b4f service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:45.888948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d359ac-c127-4c64-9aac-7ede8e9e6b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:45.889270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d359ac-c127-4c64-9aac-7ede8e9e6b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:45.898477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d359ac-c127-4c64-9aac-7ede8e9e6b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:45.899917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d359ac-c127-4c64-9aac-7ede8e9e6b4f service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:45.910260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 727/1461] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:48.007676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c18a4-64a8-43c0-90b8-dbb82dde6456 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:48.008239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c18a4-64a8-43c0-90b8-dbb82dde6456 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:48.008239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c18a4-64a8-43c0-90b8-dbb82dde6456 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:48.009632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:48.015640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1bc3c-351a-4663-b161-18568a824d52 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:48.016102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1bc3c-351a-4663-b161-18568a824d52 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:48.016199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1bc3c-351a-4663-b161-18568a824d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:48.022255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=79 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:56:48.034730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-22d1bc3c-351a-4663-b161-18568a824d52 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:48.035946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-22d1bc3c-351a-4663-b161-18568a824d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:48.043715 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:48.082188 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c18a4-64a8-43c0-90b8-dbb82dde6456 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:48.087553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=73 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:56:48.094435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c18a4-64a8-43c0-90b8-dbb82dde6456 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:48.097924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a261f6e-d14e-47d6-b6f3-eb2834ebcd82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:48.098567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a261f6e-d14e-47d6-b6f3-eb2834ebcd82 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:48.098851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a261f6e-d14e-47d6-b6f3-eb2834ebcd82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:48.099131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a261f6e-d14e-47d6-b6f3-eb2834ebcd82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:48.107999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a261f6e-d14e-47d6-b6f3-eb2834ebcd82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:48.109530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a261f6e-d14e-47d6-b6f3-eb2834ebcd82 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:48.120287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 736/1464] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:50.210668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6d12b-ff8b-49e3-9280-8153a91b4269 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:50.211243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6d12b-ff8b-49e3-9280-8153a91b4269 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:50.211243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6d12b-ff8b-49e3-9280-8153a91b4269 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:50.212627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:50.218410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e70be-09fe-43fc-86ad-3246197bcb0b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:50.218802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e70be-09fe-43fc-86ad-3246197bcb0b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:50.218972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e70be-09fe-43fc-86ad-3246197bcb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:50.237441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-447e70be-09fe-43fc-86ad-3246197bcb0b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:50.238645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-447e70be-09fe-43fc-86ad-3246197bcb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:50.246367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:56:50.282439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6d12b-ff8b-49e3-9280-8153a91b4269 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:50.293141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6d12b-ff8b-49e3-9280-8153a91b4269 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:50.297317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d39b8-6636-4619-b39f-859c75922aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:50.297953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d39b8-6636-4619-b39f-859c75922aea service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:50.298243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d39b8-6636-4619-b39f-859c75922aea service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:50.298525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d39b8-6636-4619-b39f-859c75922aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:50.307599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d39b8-6636-4619-b39f-859c75922aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:50.309038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d39b8-6636-4619-b39f-859c75922aea service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:50.319221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 730/1467] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:56:52.398285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9d633-3d6b-461e-92f5-974b398bbdd8 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:52.398849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9d633-3d6b-461e-92f5-974b398bbdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:52.398849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9d633-3d6b-461e-92f5-974b398bbdd8 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:52.400224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 18:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:56:52.405470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f450cc-5ba5-4837-b5ac-0a4277f06733 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:52.405970 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f450cc-5ba5-4837-b5ac-0a4277f06733 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:52.406078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f450cc-5ba5-4837-b5ac-0a4277f06733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:52.423500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-54f450cc-5ba5-4837-b5ac-0a4277f06733 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:56:52.424696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-54f450cc-5ba5-4837-b5ac-0a4277f06733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:56:52.432430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 18:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 18:56:52.469823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9d633-3d6b-461e-92f5-974b398bbdd8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:56:52.480408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9d633-3d6b-461e-92f5-974b398bbdd8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:56:52.483934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d481c-4666-46dd-a9d3-5b673b27986a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:56:52.484566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d481c-4666-46dd-a9d3-5b673b27986a service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:56:52.484846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d481c-4666-46dd-a9d3-5b673b27986a service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:56:52.485144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d481c-4666-46dd-a9d3-5b673b27986a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:56:52.493894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d481c-4666-46dd-a9d3-5b673b27986a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:56:52.495306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d481c-4666-46dd-a9d3-5b673b27986a service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:56:52.505040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 739/1470] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:57:02.418443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=11 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:57:02.500905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=14 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:57:12.288741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e6045-99be-43c1-b969-b05508ab9404 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:12.289399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e6045-99be-43c1-b969-b05508ab9404 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:12.289399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e6045-99be-43c1-b969-b05508ab9404 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:12.290723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 732/1471] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:57:12.296751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844d6b1-afc0-4ba9-b129-6ecdd9871dba None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:12.297186 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844d6b1-afc0-4ba9-b129-6ecdd9871dba None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:12.297356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844d6b1-afc0-4ba9-b129-6ecdd9871dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:12.315007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8844d6b1-afc0-4ba9-b129-6ecdd9871dba None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:57:12.316207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8844d6b1-afc0-4ba9-b129-6ecdd9871dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:57:12.324373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 740/1472] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:57:12.372815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22a265-7084-48ab-9608-c8642288f8fb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:12.373128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22a265-7084-48ab-9608-c8642288f8fb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:12.373420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22a265-7084-48ab-9608-c8642288f8fb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:12.374805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 733/1473] 10.222.0.70 () {54 vars in 962 bytes} [Tue Jun 21 18:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:57:12.379535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a15a3-bb02-4eb8-b251-0f1ea7584a89 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:12.379828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a15a3-bb02-4eb8-b251-0f1ea7584a89 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:12.380108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a15a3-bb02-4eb8-b251-0f1ea7584a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:12.398351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6f1a15a3-bb02-4eb8-b251-0f1ea7584a89 None None] MFA Rules not processed for user `915704f26c814eaa9e6c2a68533a9c66`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:57:12.399548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6f1a15a3-bb02-4eb8-b251-0f1ea7584a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:57:12.406598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 741/1474] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Jun 21 18:57:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:57:12.414005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22a265-7084-48ab-9608-c8642288f8fb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:57:12.424703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22a265-7084-48ab-9608-c8642288f8fb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:57:12.428191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e403cc2-29b9-4665-af46-9f02b6c6bee0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915704f26c814eaa9e6c2a68533a9c66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:57:12.428823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e403cc2-29b9-4665-af46-9f02b6c6bee0 service glance] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:12.429146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e403cc2-29b9-4665-af46-9f02b6c6bee0 service glance] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:12.429432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e403cc2-29b9-4665-af46-9f02b6c6bee0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:12.438484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e403cc2-29b9-4665-af46-9f02b6c6bee0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:57:12.440177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e403cc2-29b9-4665-af46-9f02b6c6bee0 service glance] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:57:12.449952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 734/1475] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:57:14.590451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b94c64-02ce-4b36-89f4-76b42e847df3 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:14.590967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b94c64-02ce-4b36-89f4-76b42e847df3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:14.590967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b94c64-02ce-4b36-89f4-76b42e847df3 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:14.592805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 742/1476] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:57:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:57:14.597900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f47de-4b88-486c-b429-ffc6cae8ff87 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:14.598358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f47de-4b88-486c-b429-ffc6cae8ff87 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:14.598502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f47de-4b88-486c-b429-ffc6cae8ff87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:14.616617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-264f47de-4b88-486c-b429-ffc6cae8ff87 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:57:14.617856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-264f47de-4b88-486c-b429-ffc6cae8ff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:57:14.625584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 735/1477] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:57:14.648277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b94c64-02ce-4b36-89f4-76b42e847df3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:57:14.661276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b94c64-02ce-4b36-89f4-76b42e847df3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:57:14.665651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef2975-2f5f-4327-b9e4-1df42ef29c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:57:14.666292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef2975-2f5f-4327-b9e4-1df42ef29c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:14.666578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef2975-2f5f-4327-b9e4-1df42ef29c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:14.666857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef2975-2f5f-4327-b9e4-1df42ef29c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:14.667581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef2975-2f5f-4327-b9e4-1df42ef29c17 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:57:14.669367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-dbef2975-2f5f-4327-b9e4-1df42ef29c17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:57:14.669367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jun 21 18:57:14.669367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: Jun 21 18:57:14.671058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef2975-2f5f-4327-b9e4-1df42ef29c17 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:57:14.690451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 743/1478] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jun 21 18:57:14 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:57:16.758128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7979bd4-394f-4369-910e-361bc6c0cc24 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:16.758705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7979bd4-394f-4369-910e-361bc6c0cc24 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:16.758705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7979bd4-394f-4369-910e-361bc6c0cc24 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:16.760092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 736/1479] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:57:16.765989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb44f1-242f-472c-83d9-74aec2b2ecb1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:16.766427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb44f1-242f-472c-83d9-74aec2b2ecb1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:16.766615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb44f1-242f-472c-83d9-74aec2b2ecb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:16.785566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-69fb44f1-242f-472c-83d9-74aec2b2ecb1 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:57:16.786783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-69fb44f1-242f-472c-83d9-74aec2b2ecb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:57:16.795393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 744/1480] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:57:16.839076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7979bd4-394f-4369-910e-361bc6c0cc24 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:57:16.856038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7979bd4-394f-4369-910e-361bc6c0cc24 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:57:16.862237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fa3fb-52e0-4f3a-a5f9-45176a906cd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:57:16.862895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fa3fb-52e0-4f3a-a5f9-45176a906cd3 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:16.863183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fa3fb-52e0-4f3a-a5f9-45176a906cd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:16.863467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fa3fb-52e0-4f3a-a5f9-45176a906cd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:16.872180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fa3fb-52e0-4f3a-a5f9-45176a906cd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:57:16.873847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fa3fb-52e0-4f3a-a5f9-45176a906cd3 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:57:16.884108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 737/1481] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:57:19.117910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e817160-94a6-47b0-8f57-dc0fe577baee None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:19.118501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e817160-94a6-47b0-8f57-dc0fe577baee None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:19.118501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e817160-94a6-47b0-8f57-dc0fe577baee None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:19.119868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 745/1482] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:57:19.125774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb466a2-dc8d-4112-bc04-718a6eb363ad None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:19.126237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb466a2-dc8d-4112-bc04-718a6eb363ad None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:19.126371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb466a2-dc8d-4112-bc04-718a6eb363ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:19.144648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3bb466a2-dc8d-4112-bc04-718a6eb363ad None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:57:19.145895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3bb466a2-dc8d-4112-bc04-718a6eb363ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:57:19.153742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 738/1483] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:57:19.190093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e817160-94a6-47b0-8f57-dc0fe577baee None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:57:19.201757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e817160-94a6-47b0-8f57-dc0fe577baee None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:57:19.205323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df25b3-7c95-4779-af7e-338a74a3dcf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:57:19.205956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df25b3-7c95-4779-af7e-338a74a3dcf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:19.206244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df25b3-7c95-4779-af7e-338a74a3dcf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:19.206524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df25b3-7c95-4779-af7e-338a74a3dcf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:19.215830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df25b3-7c95-4779-af7e-338a74a3dcf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:57:19.217322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df25b3-7c95-4779-af7e-338a74a3dcf1 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:57:19.226653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 746/1484] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:57:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:57:21.322937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d2919-b88f-477e-8278-724c6e1a02bd None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:21.323493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d2919-b88f-477e-8278-724c6e1a02bd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:21.323493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d2919-b88f-477e-8278-724c6e1a02bd None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:21.324877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 739/1485] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:57:21.330860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9478a97-aff3-4770-bec7-02f4fe458f4a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:21.331290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9478a97-aff3-4770-bec7-02f4fe458f4a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:21.331433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9478a97-aff3-4770-bec7-02f4fe458f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:21.349877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c9478a97-aff3-4770-bec7-02f4fe458f4a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:57:21.351073 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c9478a97-aff3-4770-bec7-02f4fe458f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:57:21.358937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 747/1486] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:57:21.397416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d2919-b88f-477e-8278-724c6e1a02bd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:57:21.408058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d2919-b88f-477e-8278-724c6e1a02bd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:57:21.411600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c369ef5-bd8e-4274-ae95-cce35e028799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:57:21.412237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c369ef5-bd8e-4274-ae95-cce35e028799 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:21.412519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c369ef5-bd8e-4274-ae95-cce35e028799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:21.412796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c369ef5-bd8e-4274-ae95-cce35e028799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:21.421710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c369ef5-bd8e-4274-ae95-cce35e028799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:57:21.423138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c369ef5-bd8e-4274-ae95-cce35e028799 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:57:21.432386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 740/1487] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:57:23.564996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cbd2a-061d-4f18-9925-16a150a54c6d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:23.565579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cbd2a-061d-4f18-9925-16a150a54c6d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:23.565579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cbd2a-061d-4f18-9925-16a150a54c6d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:23.566963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 748/1488] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:57:23.572159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb5ba0-f58f-4d8f-8582-2485a50b2ab9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:23.572589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb5ba0-f58f-4d8f-8582-2485a50b2ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:23.572732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb5ba0-f58f-4d8f-8582-2485a50b2ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:23.590839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-84bb5ba0-f58f-4d8f-8582-2485a50b2ab9 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:57:23.592018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-84bb5ba0-f58f-4d8f-8582-2485a50b2ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:57:23.599736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 741/1489] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:57:23.640695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cbd2a-061d-4f18-9925-16a150a54c6d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:57:23.651581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cbd2a-061d-4f18-9925-16a150a54c6d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:57:23.655240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26888e7-0024-4a41-9a98-78a99cf731a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:57:23.655880 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26888e7-0024-4a41-9a98-78a99cf731a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:23.656169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26888e7-0024-4a41-9a98-78a99cf731a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:23.656449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26888e7-0024-4a41-9a98-78a99cf731a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:23.665650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26888e7-0024-4a41-9a98-78a99cf731a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:57:23.667079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26888e7-0024-4a41-9a98-78a99cf731a0 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:57:23.676505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 749/1490] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:57:25.908249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c6559-b80f-4444-b043-04cb4224b16d None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:25.908822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c6559-b80f-4444-b043-04cb4224b16d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:25.908822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c6559-b80f-4444-b043-04cb4224b16d None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:25.910231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 742/1491] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jun 21 18:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:57:25.915423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81404-a83d-48a1-afc3-2e7212ffbf8b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:25.915881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81404-a83d-48a1-afc3-2e7212ffbf8b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:25.915992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a81404-a83d-48a1-afc3-2e7212ffbf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:25.933586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-94a81404-a83d-48a1-afc3-2e7212ffbf8b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:57:25.935366 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-94a81404-a83d-48a1-afc3-2e7212ffbf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:57:25.943452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 750/1492] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jun 21 18:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:57:25.982472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c6559-b80f-4444-b043-04cb4224b16d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:57:25.993278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c6559-b80f-4444-b043-04cb4224b16d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:57:25.996716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0d1e6-85cd-4cfd-8a0e-40ca712b8c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:57:25.997940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0d1e6-85cd-4cfd-8a0e-40ca712b8c81 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:57:25.998228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0d1e6-85cd-4cfd-8a0e-40ca712b8c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:57:25.998508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0d1e6-85cd-4cfd-8a0e-40ca712b8c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:57:26.007616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0d1e6-85cd-4cfd-8a0e-40ca712b8c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:57:26.009052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0d1e6-85cd-4cfd-8a0e-40ca712b8c81 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:57:26.019223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 743/1493] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:57:35.929338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=33 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:57:36.014956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=33 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:58:57.678326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa58d3-3930-4925-b120-4acafaede04b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:57.678896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa58d3-3930-4925-b120-4acafaede04b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:57.678896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa58d3-3930-4925-b120-4acafaede04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:57.696848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-44aa58d3-3930-4925-b120-4acafaede04b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:58:57.698592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-44aa58d3-3930-4925-b120-4acafaede04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:58:57.706537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 751/1494] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 18:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:58:57.716297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53d6b9-abed-4ef0-b096-b6f5bb4c0b22 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:57.716827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53d6b9-abed-4ef0-b096-b6f5bb4c0b22 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:57.716942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53d6b9-abed-4ef0-b096-b6f5bb4c0b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:57.734754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-dc53d6b9-abed-4ef0-b096-b6f5bb4c0b22 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:58:57.736010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-dc53d6b9-abed-4ef0-b096-b6f5bb4c0b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:58:57.743719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 744/1495] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 18:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:58:57.748335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa58d3-3930-4925-b120-4acafaede04b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:57.757980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa58d3-3930-4925-b120-4acafaede04b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:57.762915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:57.763562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:57.763847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:57.764126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:57.764849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 18:58:57.765195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:57.766205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:58:57.766205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jun 21 18:58:57.766205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: Jun 21 18:58:57.767079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:57.779232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 752/1496] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 18:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 18:58:57.787231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53d6b9-abed-4ef0-b096-b6f5bb4c0b22 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:57.796413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53d6b9-abed-4ef0-b096-b6f5bb4c0b22 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:57.800937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0c476-2142-48b0-85aa-0987059fb0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:57.801614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0c476-2142-48b0-85aa-0987059fb0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:57.801902 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0c476-2142-48b0-85aa-0987059fb0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:57.802181 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0c476-2142-48b0-85aa-0987059fb0c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:57.802883 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0c476-2142-48b0-85aa-0987059fb0c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:57.804652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-bfd0c476-2142-48b0-85aa-0987059fb0c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:58:57.804652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 21 18:58:57.804652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: Jun 21 18:58:57.805890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0c476-2142-48b0-85aa-0987059fb0c1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:57.842397 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 745/1497] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 18:58:57 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:58:57.846750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:57.856661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb575e6d-4da6-4184-b0b9-271c0ca84be3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:57.911747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfdc81-666e-4f6e-8ce1-1f92cd46948f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:57.912445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfdc81-666e-4f6e-8ce1-1f92cd46948f admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:57.912725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfdc81-666e-4f6e-8ce1-1f92cd46948f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:57.912999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfdc81-666e-4f6e-8ce1-1f92cd46948f admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:57.913709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfdc81-666e-4f6e-8ce1-1f92cd46948f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:57.914715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-80bfdc81-666e-4f6e-8ce1-1f92cd46948f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:58:57.914715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jun 21 18:58:57.914715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: Jun 21 18:58:57.915911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfdc81-666e-4f6e-8ce1-1f92cd46948f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:57.937039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 753/1498] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 18:58:57 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 18:58:57.941599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0c476-2142-48b0-85aa-0987059fb0c1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:57.952444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0c476-2142-48b0-85aa-0987059fb0c1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:57.956938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad38d3d-c1a7-4f81-8451-8409f432410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:57.957631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad38d3d-c1a7-4f81-8451-8409f432410e admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:57.957923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad38d3d-c1a7-4f81-8451-8409f432410e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:57.958204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad38d3d-c1a7-4f81-8451-8409f432410e admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:57.958932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad38d3d-c1a7-4f81-8451-8409f432410e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:57.959953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-9ad38d3d-c1a7-4f81-8451-8409f432410e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:58:57.959953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 21 18:58:57.959953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: Jun 21 18:58:57.960845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad38d3d-c1a7-4f81-8451-8409f432410e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:57.974229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 746/1499] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 18:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:58:57.978233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfdc81-666e-4f6e-8ce1-1f92cd46948f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:57.988039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfdc81-666e-4f6e-8ce1-1f92cd46948f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:57.992649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981e7a0-0637-437c-8675-b1c22d89653e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:57.993331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981e7a0-0637-437c-8675-b1c22d89653e admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:57.993619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981e7a0-0637-437c-8675-b1c22d89653e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:57.993899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981e7a0-0637-437c-8675-b1c22d89653e admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:57.994617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981e7a0-0637-437c-8675-b1c22d89653e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:57.995618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-e981e7a0-0637-437c-8675-b1c22d89653e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:58:57.995618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jun 21 18:58:57.995618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: Jun 21 18:58:57.996494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981e7a0-0637-437c-8675-b1c22d89653e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:58.011372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 754/1500] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 18:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 18:58:58.015452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad38d3d-c1a7-4f81-8451-8409f432410e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:58.025798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad38d3d-c1a7-4f81-8451-8409f432410e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:58.030989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6adce-cda8-482b-9969-29f5095d30df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:58.031879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6adce-cda8-482b-9969-29f5095d30df admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:58.032172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6adce-cda8-482b-9969-29f5095d30df admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:58.032452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6adce-cda8-482b-9969-29f5095d30df admin admin] PATH_INFO: `/v3/projects/633c7d2bf1fe499aa38e8fda9602faee/users/b1a7aaa02f974f848837648eaf68e4ed/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:58.047781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6adce-cda8-482b-9969-29f5095d30df admin admin] RBAC: Authorizing `identity:create_grant(project_id=633c7d2bf1fe499aa38e8fda9602faee, user_id=b1a7aaa02f974f848837648eaf68e4ed, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:58.048907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-cee6adce-cda8-482b-9969-29f5095d30df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:58:58.048907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 21 18:58:58.048907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: Jun 21 18:58:58.050915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6adce-cda8-482b-9969-29f5095d30df admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:58.064113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 747/1501] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 18:58:58 2022] PUT /identity/v3/projects/633c7d2bf1fe499aa38e8fda9602faee/users/b1a7aaa02f974f848837648eaf68e4ed/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:58:58.076310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec725d-389b-45be-abd1-4919491a9baa None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:58.076604 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec725d-389b-45be-abd1-4919491a9baa None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:58.076886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec725d-389b-45be-abd1-4919491a9baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:58.104271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-71ec725d-389b-45be-abd1-4919491a9baa None None] MFA Rules not processed for user `b1a7aaa02f974f848837648eaf68e4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:58:58.105527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-71ec725d-389b-45be-abd1-4919491a9baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:58:58.169864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 755/1502] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 18:58:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:58:58.183984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdf88f-5e4f-409c-a64b-88ad8ed4cd47 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:58.184275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdf88f-5e4f-409c-a64b-88ad8ed4cd47 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:58.184555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdf88f-5e4f-409c-a64b-88ad8ed4cd47 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:58.185985 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 748/1503] 10.222.0.70 () {54 vars in 963 bytes} [Tue Jun 21 18:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:58:58.191130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a013f78-cce5-4c13-9d3c-0f7a2468b6fe None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:58.191369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a013f78-cce5-4c13-9d3c-0f7a2468b6fe None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:58.191659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a013f78-cce5-4c13-9d3c-0f7a2468b6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:58.209909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0a013f78-cce5-4c13-9d3c-0f7a2468b6fe None None] MFA Rules not processed for user `5b8161446e1344108f8283d46da23ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:58:58.211115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0a013f78-cce5-4c13-9d3c-0f7a2468b6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:58:58.253867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:58:58.262776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 756/1504] 10.222.0.70 () {58 vars in 1060 bytes} [Tue Jun 21 18:58:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:58:58.270226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdf88f-5e4f-409c-a64b-88ad8ed4cd47 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:58.280807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdf88f-5e4f-409c-a64b-88ad8ed4cd47 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:58.284349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e863e-f3e8-4640-935b-5599dc0977af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:58.284980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e863e-f3e8-4640-935b-5599dc0977af service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:58.285322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e863e-f3e8-4640-935b-5599dc0977af service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:58.285605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e863e-f3e8-4640-935b-5599dc0977af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:58.294010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e863e-f3e8-4640-935b-5599dc0977af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:58.295445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e863e-f3e8-4640-935b-5599dc0977af service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:58.304364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 749/1505] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:58:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:58:58.347030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca27e9a-99c4-45d9-9524-7b28dda8fdbf None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:58.347318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca27e9a-99c4-45d9-9524-7b28dda8fdbf None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:58.347596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca27e9a-99c4-45d9-9524-7b28dda8fdbf None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:58.348897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 757/1506] 10.222.0.70 () {52 vars in 829 bytes} [Tue Jun 21 18:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 18:58:58.352642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e863e-f3e8-4640-935b-5599dc0977af service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:58.415293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e863e-f3e8-4640-935b-5599dc0977af service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:58.420305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:58.421193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:58.421475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:58.421751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] PATH_INFO: `/v3/users/b1a7aaa02f974f848837648eaf68e4ed` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:58.422921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a7aaa02f974f848837648eaf68e4ed)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:58.423936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:58:58.423936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 21 18:58:58.423936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: Jun 21 18:58:58.425171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:58.490317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a7aaa02f974f848837648eaf68e4ed'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:58.508869 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a7aaa02f974f848837648eaf68e4ed'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:58.518382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a7aaa02f974f848837648eaf68e4ed'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:58.527057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:58:58.531942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a7aaa02f974f848837648eaf68e4ed'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:58.533260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a7aaa02f974f848837648eaf68e4ed'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:58.539643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 750/1507] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 18:58:58 2022] DELETE /identity/v3/users/b1a7aaa02f974f848837648eaf68e4ed => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:58:58.544605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea7e72-4d15-4e44-a5c9-86602235943b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:58.544904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea7e72-4d15-4e44-a5c9-86602235943b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:58.545328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea7e72-4d15-4e44-a5c9-86602235943b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:58.563075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d4ea7e72-4d15-4e44-a5c9-86602235943b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 18:58:58.564279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d4ea7e72-4d15-4e44-a5c9-86602235943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:58:58.623047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 758/1508] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 18:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 18:58:58.633288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:58.634708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:58:58.691823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cf311-241f-4882-9d47-3e21787649c6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:58.695823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d320da-9231-4f0b-875a-97294b43e130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:58.696471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d320da-9231-4f0b-875a-97294b43e130 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:58.696756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d320da-9231-4f0b-875a-97294b43e130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:58.697038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d320da-9231-4f0b-875a-97294b43e130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:58.705932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d320da-9231-4f0b-875a-97294b43e130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:58.707372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d320da-9231-4f0b-875a-97294b43e130 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:58.716620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 751/1509] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 18:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 18:58:59.080677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea7e72-4d15-4e44-a5c9-86602235943b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 18:58:59.082316 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d4ea7e72-4d15-4e44-a5c9-86602235943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 18:58:59.094607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea7e72-4d15-4e44-a5c9-86602235943b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 18:58:59.100030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 18:58:59.101061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 18:58:59.101463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 18:58:59.104847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] PATH_INFO: `/v3/projects/633c7d2bf1fe499aa38e8fda9602faee` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 18:58:59.105559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=633c7d2bf1fe499aa38e8fda9602faee)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 18:58:59.106662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 18:58:59.106662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jun 21 18:58:59.106662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: Jun 21 18:58:59.108079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 18:58:59.173810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633c7d2bf1fe499aa38e8fda9602faee', 'request_id': 'req-ab31baa1-fde8-413d-a193-8eecc76b08e1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:59.187181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633c7d2bf1fe499aa38e8fda9602faee', 'request_id': 'req-ab31baa1-fde8-413d-a193-8eecc76b08e1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:59.197350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633c7d2bf1fe499aa38e8fda9602faee', 'request_id': 'req-ab31baa1-fde8-413d-a193-8eecc76b08e1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:59.209296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] The token cache is being invalidate because project 633c7d2bf1fe499aa38e8fda9602faee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 18:58:59.209688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:59.210343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab31baa1-fde8-413d-a193-8eecc76b08e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 18:58:59.211989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 759/1510] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 18:58:59 2022] DELETE /identity/v3/projects/633c7d2bf1fe499aa38e8fda9602faee => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 18:59:08.712504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=29 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 18:59:09.205984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:00:35.170942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eb285-39a6-464c-bda1-616f16151e6b None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:35.171658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eb285-39a6-464c-bda1-616f16151e6b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:35.171658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eb285-39a6-464c-bda1-616f16151e6b None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:35.172966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 19:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 19:00:35.178626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e075ac-499f-43f8-bbde-4aeb9b071b74 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:35.179242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e075ac-499f-43f8-bbde-4aeb9b071b74 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:35.179242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e075ac-499f-43f8-bbde-4aeb9b071b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:35.199727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-68e075ac-499f-43f8-bbde-4aeb9b071b74 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:00:35.201049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-68e075ac-499f-43f8-bbde-4aeb9b071b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:35.293109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 19:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:00:35.452248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eb285-39a6-464c-bda1-616f16151e6b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:00:35.454878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2d4eb285-39a6-464c-bda1-616f16151e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:35.543401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eb285-39a6-464c-bda1-616f16151e6b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:00:35.549263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0b2bc-ad94-49f8-a948-4f7baa3affac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:00:35.550042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0b2bc-ad94-49f8-a948-4f7baa3affac service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:35.550418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0b2bc-ad94-49f8-a948-4f7baa3affac service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:35.550788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0b2bc-ad94-49f8-a948-4f7baa3affac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:35.560244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0b2bc-ad94-49f8-a948-4f7baa3affac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:00:35.562646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0b2bc-ad94-49f8-a948-4f7baa3affac service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:00:35.575047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 753/1513] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:00:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:00:36.487310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877ccb3-2442-41f1-8cf9-edde4abe04ee None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:36.487883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877ccb3-2442-41f1-8cf9-edde4abe04ee None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:36.487883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877ccb3-2442-41f1-8cf9-edde4abe04ee None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:36.489264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 19:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:00:36.495131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa269c-b70e-4798-bf51-c5d7c1187743 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:36.495881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa269c-b70e-4798-bf51-c5d7c1187743 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:36.496042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa269c-b70e-4798-bf51-c5d7c1187743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:36.515079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a3aa269c-b70e-4798-bf51-c5d7c1187743 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:00:36.516307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a3aa269c-b70e-4798-bf51-c5d7c1187743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:36.524759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 19:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:00:36.677958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877ccb3-2442-41f1-8cf9-edde4abe04ee None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:00:36.694447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877ccb3-2442-41f1-8cf9-edde4abe04ee None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:00:36.700367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771c981-09cc-4413-8354-7fdd03c427d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:00:36.701012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771c981-09cc-4413-8354-7fdd03c427d0 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:36.701333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771c981-09cc-4413-8354-7fdd03c427d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:36.701609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771c981-09cc-4413-8354-7fdd03c427d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:36.710285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771c981-09cc-4413-8354-7fdd03c427d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:00:36.711950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771c981-09cc-4413-8354-7fdd03c427d0 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:00:36.722623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 762/1516] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:00:37.767999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec545cd-daa6-48d8-aa83-1ebc87b2877c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:37.768603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec545cd-daa6-48d8-aa83-1ebc87b2877c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:37.768603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec545cd-daa6-48d8-aa83-1ebc87b2877c None None] PATH_INFO: `/v3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:37.770135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 755/1517] 10.222.0.18 () {54 vars in 925 bytes} [Tue Jun 21 19:00:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:00:37.777876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a31d1-b763-4199-8e13-ba3f409fc2ea None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:37.778546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a31d1-b763-4199-8e13-ba3f409fc2ea None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:37.778849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a31d1-b763-4199-8e13-ba3f409fc2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:37.815078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fc3a31d1-b763-4199-8e13-ba3f409fc2ea None None] MFA Rules not processed for user `f0e3fc619dc74a0591d8ad14369ae673`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:00:37.817510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fc3a31d1-b763-4199-8e13-ba3f409fc2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:37.943901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 763/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jun 21 19:00:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:00:41.758709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05f32-18f1-4568-91b2-1a5367c58e8e None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:41.759349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05f32-18f1-4568-91b2-1a5367c58e8e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:41.759493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05f32-18f1-4568-91b2-1a5367c58e8e None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:41.760847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 756/1519] 10.222.0.70 () {54 vars in 960 bytes} [Tue Jun 21 19:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 19:00:41.766143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a1c5d-35bd-4a93-aa18-848cf25874f3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:41.766728 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a1c5d-35bd-4a93-aa18-848cf25874f3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:41.766728 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a1c5d-35bd-4a93-aa18-848cf25874f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:41.784409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8f9a1c5d-35bd-4a93-aa18-848cf25874f3 None None] MFA Rules not processed for user `f0e3fc619dc74a0591d8ad14369ae673`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:00:41.785660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8f9a1c5d-35bd-4a93-aa18-848cf25874f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:41.794660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 764/1520] 10.222.0.70 () {58 vars in 1057 bytes} [Tue Jun 21 19:00:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:00:41.802709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca05f32-18f1-4568-91b2-1a5367c58e8e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:00:41.817477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca05f32-18f1-4568-91b2-1a5367c58e8e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:00:41.822739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1af87c-6889-4ddd-b6c2-eb47b70c0170 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:00:41.823383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1af87c-6889-4ddd-b6c2-eb47b70c0170 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:41.823668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1af87c-6889-4ddd-b6c2-eb47b70c0170 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:41.823945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1af87c-6889-4ddd-b6c2-eb47b70c0170 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:41.832622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1af87c-6889-4ddd-b6c2-eb47b70c0170 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:00:41.834278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1af87c-6889-4ddd-b6c2-eb47b70c0170 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:00:41.844385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 757/1521] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:00:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:00:48.011352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45142f-3a68-425c-a819-446035fc45cf None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:48.011940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45142f-3a68-425c-a819-446035fc45cf None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:48.011940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45142f-3a68-425c-a819-446035fc45cf None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:48.013439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 19:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:00:48.018533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd70fde-0dbc-412a-9d88-5f97976f9d4c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:48.019070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd70fde-0dbc-412a-9d88-5f97976f9d4c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:48.019217 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd70fde-0dbc-412a-9d88-5f97976f9d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:48.036899 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-acd70fde-0dbc-412a-9d88-5f97976f9d4c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:00:48.038137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-acd70fde-0dbc-412a-9d88-5f97976f9d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:48.045959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 19:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:00:48.201061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45142f-3a68-425c-a819-446035fc45cf None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:00:48.216515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45142f-3a68-425c-a819-446035fc45cf None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:00:48.221931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa852d-898d-4a24-b6e9-bcdb28d8c76e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:00:48.222584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa852d-898d-4a24-b6e9-bcdb28d8c76e service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:48.222877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa852d-898d-4a24-b6e9-bcdb28d8c76e service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:48.223160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa852d-898d-4a24-b6e9-bcdb28d8c76e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:48.231982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa852d-898d-4a24-b6e9-bcdb28d8c76e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:00:48.233649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa852d-898d-4a24-b6e9-bcdb28d8c76e service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:00:48.244084 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 766/1524] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:00:49.085591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a78d7f-ccca-4eaf-accf-619cc0acae7f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:49.086792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a78d7f-ccca-4eaf-accf-619cc0acae7f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:49.086792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a78d7f-ccca-4eaf-accf-619cc0acae7f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:49.088124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 19:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:00:49.093491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70eb09-0980-4d20-990b-e7a421928974 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:49.094045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70eb09-0980-4d20-990b-e7a421928974 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:49.094140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70eb09-0980-4d20-990b-e7a421928974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:49.112469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-cc70eb09-0980-4d20-990b-e7a421928974 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:00:49.113722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cc70eb09-0980-4d20-990b-e7a421928974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:49.121804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 19:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:00:49.276393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a78d7f-ccca-4eaf-accf-619cc0acae7f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:00:49.293730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a78d7f-ccca-4eaf-accf-619cc0acae7f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:00:49.299173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649a7f6-8a12-4699-b941-2c7675223be1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:00:49.299831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649a7f6-8a12-4699-b941-2c7675223be1 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:49.300115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649a7f6-8a12-4699-b941-2c7675223be1 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:49.300390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649a7f6-8a12-4699-b941-2c7675223be1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:49.309540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649a7f6-8a12-4699-b941-2c7675223be1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:00:49.311207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649a7f6-8a12-4699-b941-2c7675223be1 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:00:49.322248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 760/1527] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:00:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:00:53.307997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2064c-f2b1-4b74-8610-8c4d6e6fceb9 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:53.308556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2064c-f2b1-4b74-8610-8c4d6e6fceb9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:53.308556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2064c-f2b1-4b74-8610-8c4d6e6fceb9 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:53.310032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 19:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:00:53.316449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b6453-8201-45a2-87ed-49d06e4a2d8a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:53.316960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b6453-8201-45a2-87ed-49d06e4a2d8a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:53.317082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b6453-8201-45a2-87ed-49d06e4a2d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:53.337888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a20b6453-8201-45a2-87ed-49d06e4a2d8a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:00:53.339116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a20b6453-8201-45a2-87ed-49d06e4a2d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:53.347006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 19:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:00:53.384315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2064c-f2b1-4b74-8610-8c4d6e6fceb9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:00:53.386134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-57d2064c-f2b1-4b74-8610-8c4d6e6fceb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:53.444038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2064c-f2b1-4b74-8610-8c4d6e6fceb9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:00:53.448251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade1739-b27f-4bf5-b02e-8f2aeee47bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:00:53.448902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade1739-b27f-4bf5-b02e-8f2aeee47bcf service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:53.449222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade1739-b27f-4bf5-b02e-8f2aeee47bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:53.449501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade1739-b27f-4bf5-b02e-8f2aeee47bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:53.458858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade1739-b27f-4bf5-b02e-8f2aeee47bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:00:53.460289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade1739-b27f-4bf5-b02e-8f2aeee47bcf service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:00:53.465418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:00:53.470864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 769/1530] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:00:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:00:55.529189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998db8e-bf42-47e9-bdea-890fdadf7846 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:55.529755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998db8e-bf42-47e9-bdea-890fdadf7846 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:55.529755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998db8e-bf42-47e9-bdea-890fdadf7846 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:55.531171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 19:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:00:55.536605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14048d9-7741-4cb2-86de-c3333c34c36e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:55.537069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14048d9-7741-4cb2-86de-c3333c34c36e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:55.537312 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14048d9-7741-4cb2-86de-c3333c34c36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:55.556049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d14048d9-7741-4cb2-86de-c3333c34c36e None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:00:55.557269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d14048d9-7741-4cb2-86de-c3333c34c36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:55.565046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 19:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:00:55.602938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998db8e-bf42-47e9-bdea-890fdadf7846 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:00:55.613873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998db8e-bf42-47e9-bdea-890fdadf7846 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:00:55.617366 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0baf3b-15d1-47b8-9d88-7a1109fda0f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:00:55.618005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0baf3b-15d1-47b8-9d88-7a1109fda0f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:55.618287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0baf3b-15d1-47b8-9d88-7a1109fda0f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:55.618560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0baf3b-15d1-47b8-9d88-7a1109fda0f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:55.627350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0baf3b-15d1-47b8-9d88-7a1109fda0f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:00:55.628815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0baf3b-15d1-47b8-9d88-7a1109fda0f5 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:00:55.638149 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 763/1533] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:00:59.246109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e225975-6e1b-42a0-a648-02e05d984f0c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:59.246686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e225975-6e1b-42a0-a648-02e05d984f0c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:59.246686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e225975-6e1b-42a0-a648-02e05d984f0c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:59.248053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 19:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:00:59.253452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25e843-65a6-4fd8-8178-3b17815d6e85 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:59.253910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25e843-65a6-4fd8-8178-3b17815d6e85 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:59.254023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25e843-65a6-4fd8-8178-3b17815d6e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:59.261006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=62 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:00:59.279468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3d25e843-65a6-4fd8-8178-3b17815d6e85 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:00:59.280821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3d25e843-65a6-4fd8-8178-3b17815d6e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:59.288921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 19:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:00:59.599978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e225975-6e1b-42a0-a648-02e05d984f0c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:00:59.601750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1e225975-6e1b-42a0-a648-02e05d984f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:00:59.660608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e225975-6e1b-42a0-a648-02e05d984f0c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:00:59.664266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a5ef3-f3c5-44c0-b0ef-8eefd8c26d1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:00:59.664915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5ef3-f3c5-44c0-b0ef-8eefd8c26d1b service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:00:59.665226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5ef3-f3c5-44c0-b0ef-8eefd8c26d1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:00:59.665618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5ef3-f3c5-44c0-b0ef-8eefd8c26d1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:00:59.674827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5ef3-f3c5-44c0-b0ef-8eefd8c26d1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:00:59.676287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5ef3-f3c5-44c0-b0ef-8eefd8c26d1b service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:00:59.685712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 772/1536] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:00:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:01:01.780863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580ec3-9acd-47d3-bfaa-ed8e178eb0ab None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:01:01.781480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580ec3-9acd-47d3-bfaa-ed8e178eb0ab None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:01:01.781480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580ec3-9acd-47d3-bfaa-ed8e178eb0ab None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:01:01.782900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 19:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:01:01.788069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f269d-c928-4b01-90a6-becc8a42efeb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:01:01.788523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f269d-c928-4b01-90a6-becc8a42efeb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:01:01.788705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f269d-c928-4b01-90a6-becc8a42efeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:01:01.806730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-591f269d-c928-4b01-90a6-becc8a42efeb None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:01:01.807942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-591f269d-c928-4b01-90a6-becc8a42efeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:01:01.816149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 19:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:01:02.125171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580ec3-9acd-47d3-bfaa-ed8e178eb0ab None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:01:02.126878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0f580ec3-9acd-47d3-bfaa-ed8e178eb0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:01:02.140348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580ec3-9acd-47d3-bfaa-ed8e178eb0ab None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:01:02.143999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb81ab-1025-463e-b781-857b6d63100b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:01:02.144644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb81ab-1025-463e-b781-857b6d63100b service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:01:02.144934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb81ab-1025-463e-b781-857b6d63100b service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:01:02.145253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb81ab-1025-463e-b781-857b6d63100b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:01:02.155669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb81ab-1025-463e-b781-857b6d63100b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:01:02.157157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb81ab-1025-463e-b781-857b6d63100b service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:01:02.167268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 766/1539] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:01:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:01:11.801423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=32 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:01:12.162702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=8 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:31.381684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec5cf3-f304-4715-bf60-20c093b33c0b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:31.382794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec5cf3-f304-4715-bf60-20c093b33c0b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:31.383647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec5cf3-f304-4715-bf60-20c093b33c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:31.435574 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2cec5cf3-f304-4715-bf60-20c093b33c0b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:31.437208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2cec5cf3-f304-4715-bf60-20c093b33c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:31.449345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 774/1540] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:02:31.460550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b14441-ca42-4a14-8ec7-ab67c4da6916 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:31.461644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b14441-ca42-4a14-8ec7-ab67c4da6916 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:31.462157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b14441-ca42-4a14-8ec7-ab67c4da6916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:31.506909 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b1b14441-ca42-4a14-8ec7-ab67c4da6916 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:31.508525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b1b14441-ca42-4a14-8ec7-ab67c4da6916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:31.526374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 767/1541] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:02:31.531402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec5cf3-f304-4715-bf60-20c093b33c0b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:31.545426 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec5cf3-f304-4715-bf60-20c093b33c0b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:31.554861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d76018-a2ef-44e1-8fb6-054fc235e000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:31.555923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d76018-a2ef-44e1-8fb6-054fc235e000 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:31.556424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d76018-a2ef-44e1-8fb6-054fc235e000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:31.556928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d76018-a2ef-44e1-8fb6-054fc235e000 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:31.557948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d76018-a2ef-44e1-8fb6-054fc235e000 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:02:31.558481 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d76018-a2ef-44e1-8fb6-054fc235e000 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:31.560331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d76018-a2ef-44e1-8fb6-054fc235e000 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:31.579458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 775/1542] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:02:31.588593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b14441-ca42-4a14-8ec7-ab67c4da6916 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:31.600258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b14441-ca42-4a14-8ec7-ab67c4da6916 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:31.612557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f2b5c-6769-42e5-8648-3dcaa453a1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:31.613756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f2b5c-6769-42e5-8648-3dcaa453a1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:31.614269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f2b5c-6769-42e5-8648-3dcaa453a1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:31.614794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f2b5c-6769-42e5-8648-3dcaa453a1f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:31.615722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f2b5c-6769-42e5-8648-3dcaa453a1f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:31.617945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f2b5c-6769-42e5-8648-3dcaa453a1f0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:31.671141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 768/1543] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:31 2022] POST /identity/v3/projects => generated 365 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:02:31.682337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d76018-a2ef-44e1-8fb6-054fc235e000 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:31.693763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d76018-a2ef-44e1-8fb6-054fc235e000 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:31.787702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c080a-dc17-4708-9cad-0855a1933d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:31.791255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c080a-dc17-4708-9cad-0855a1933d61 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:31.791597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c080a-dc17-4708-9cad-0855a1933d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:31.791900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c080a-dc17-4708-9cad-0855a1933d61 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:31.792754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c080a-dc17-4708-9cad-0855a1933d61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:31.794993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c080a-dc17-4708-9cad-0855a1933d61 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:31.841757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 776/1544] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:31 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:31.846515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f2b5c-6769-42e5-8648-3dcaa453a1f0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:31.882083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f2b5c-6769-42e5-8648-3dcaa453a1f0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:31.894687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583eed-f440-4d5a-a67e-654f79f6f461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:31.896297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583eed-f440-4d5a-a67e-654f79f6f461 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:31.896792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583eed-f440-4d5a-a67e-654f79f6f461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:31.897292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583eed-f440-4d5a-a67e-654f79f6f461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:31.898605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75583eed-f440-4d5a-a67e-654f79f6f461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:31.901700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75583eed-f440-4d5a-a67e-654f79f6f461 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:31.932082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 769/1545] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:31.935782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c080a-dc17-4708-9cad-0855a1933d61 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:31.951653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c080a-dc17-4708-9cad-0855a1933d61 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:31.961533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed7f25-3e61-4b2c-89b7-e1c96a0d89e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:31.962230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed7f25-3e61-4b2c-89b7-e1c96a0d89e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:31.962513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed7f25-3e61-4b2c-89b7-e1c96a0d89e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:31.962791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed7f25-3e61-4b2c-89b7-e1c96a0d89e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:31.963531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed7f25-3e61-4b2c-89b7-e1c96a0d89e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:31.965216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed7f25-3e61-4b2c-89b7-e1c96a0d89e5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:31.981975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 777/1546] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:02:31.986704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583eed-f440-4d5a-a67e-654f79f6f461 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:32.013859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583eed-f440-4d5a-a67e-654f79f6f461 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:32.034294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f7526-b771-44ef-b1d9-0b2142f13456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:32.035916 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f7526-b771-44ef-b1d9-0b2142f13456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:32.036426 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f7526-b771-44ef-b1d9-0b2142f13456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:32.036939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f7526-b771-44ef-b1d9-0b2142f13456 admin admin] PATH_INFO: `/v3/projects/500d748635b54d98b53d1f708456ec71/users/355cc6f51cc64835b059262f841c2bbc/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:32.075454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f7526-b771-44ef-b1d9-0b2142f13456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=500d748635b54d98b53d1f708456ec71, user_id=355cc6f51cc64835b059262f841c2bbc, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:32.083363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f7526-b771-44ef-b1d9-0b2142f13456 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:32.118752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 770/1547] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:31 2022] PUT /identity/v3/projects/500d748635b54d98b53d1f708456ec71/users/355cc6f51cc64835b059262f841c2bbc/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:02:32.131838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26053eb-607c-42bc-a0ef-4f162e1a917d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:32.132147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26053eb-607c-42bc-a0ef-4f162e1a917d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:32.132390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26053eb-607c-42bc-a0ef-4f162e1a917d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:32.182092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a26053eb-607c-42bc-a0ef-4f162e1a917d None None] MFA Rules not processed for user `355cc6f51cc64835b059262f841c2bbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:32.186003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a26053eb-607c-42bc-a0ef-4f162e1a917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:32.371780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21740e0a-ccb5-4b81-a480-8daf2de166f9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:32.372162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21740e0a-ccb5-4b81-a480-8daf2de166f9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:32.373137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21740e0a-ccb5-4b81-a480-8daf2de166f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:32.384535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 778/1548] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:02:32.410489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26053eb-607c-42bc-a0ef-4f162e1a917d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:32.423615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-21740e0a-ccb5-4b81-a480-8daf2de166f9 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:32.424898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-21740e0a-ccb5-4b81-a480-8daf2de166f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:32.543119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 771/1549] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:02:32.553817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580da87-8e1b-47e2-b5c6-28af1d3787f3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:32.554140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580da87-8e1b-47e2-b5c6-28af1d3787f3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:32.554469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580da87-8e1b-47e2-b5c6-28af1d3787f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:32.614080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6580da87-8e1b-47e2-b5c6-28af1d3787f3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:32.615413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6580da87-8e1b-47e2-b5c6-28af1d3787f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:32.629463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:32.638352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 772/1550] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:02:32.643804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26053eb-607c-42bc-a0ef-4f162e1a917d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:32.645573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580da87-8e1b-47e2-b5c6-28af1d3787f3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:32.656116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580da87-8e1b-47e2-b5c6-28af1d3787f3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:32.656423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d035d-6e0d-4353-9faa-f2c7e8d9931c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:32.657320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d035d-6e0d-4353-9faa-f2c7e8d9931c service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:32.657848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d035d-6e0d-4353-9faa-f2c7e8d9931c service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:32.658341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d035d-6e0d-4353-9faa-f2c7e8d9931c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:32.673915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:32.674625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:32.674901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:32.675180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:32.676518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:02:32.676837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:32.677923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 19:02:32.677923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 21 19:02:32.677923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: Jun 21 19:02:32.678825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:32.681891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d035d-6e0d-4353-9faa-f2c7e8d9931c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:32.687041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d035d-6e0d-4353-9faa-f2c7e8d9931c service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:32.696999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 773/1551] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:02:32.708533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:32.715867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 779/1552] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:32.735926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8de880-60c0-4b48-bd90-b5a6f3e1bef7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:32.762501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff33ad-4890-47a4-a08e-82e30c0eb799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:32.763188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff33ad-4890-47a4-a08e-82e30c0eb799 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:32.763475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff33ad-4890-47a4-a08e-82e30c0eb799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:32.763814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff33ad-4890-47a4-a08e-82e30c0eb799 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:32.764529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff33ad-4890-47a4-a08e-82e30c0eb799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:32.766531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff33ad-4890-47a4-a08e-82e30c0eb799 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:32.836260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 774/1553] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:32 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:32.842601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d035d-6e0d-4353-9faa-f2c7e8d9931c service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:32.884306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d035d-6e0d-4353-9faa-f2c7e8d9931c service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:33.096477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d69621-e60a-4749-95a3-e2f02ff8986d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:33.097716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d69621-e60a-4749-95a3-e2f02ff8986d admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.098223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d69621-e60a-4749-95a3-e2f02ff8986d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.098743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d69621-e60a-4749-95a3-e2f02ff8986d admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.100025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d69621-e60a-4749-95a3-e2f02ff8986d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:33.112533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d69621-e60a-4749-95a3-e2f02ff8986d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:33.178845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 780/1554] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:32 2022] POST /identity/v3/users => generated 290 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:33.186318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff33ad-4890-47a4-a08e-82e30c0eb799 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:33.237588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff33ad-4890-47a4-a08e-82e30c0eb799 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:33.260377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68260d4-7d2f-4143-8e75-ee9a4e4f6700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:33.269812 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68260d4-7d2f-4143-8e75-ee9a4e4f6700 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.270306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68260d4-7d2f-4143-8e75-ee9a4e4f6700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.271213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68260d4-7d2f-4143-8e75-ee9a4e4f6700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.272420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68260d4-7d2f-4143-8e75-ee9a4e4f6700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:33.277127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68260d4-7d2f-4143-8e75-ee9a4e4f6700 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:33.318466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc049c-e6f8-4415-b0d2-574292753c00 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.319116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc049c-e6f8-4415-b0d2-574292753c00 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.319554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 775/1555] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:33.320141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc049c-e6f8-4415-b0d2-574292753c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.331330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68260d4-7d2f-4143-8e75-ee9a4e4f6700 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:33.381124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68260d4-7d2f-4143-8e75-ee9a4e4f6700 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:33.408041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfb8c6-f40b-49a9-a4c8-d968b53f7719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:33.409119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfb8c6-f40b-49a9-a4c8-d968b53f7719 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.409580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfb8c6-f40b-49a9-a4c8-d968b53f7719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.410134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfb8c6-f40b-49a9-a4c8-d968b53f7719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.411864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e8bc049c-e6f8-4415-b0d2-574292753c00 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:33.413446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfb8c6-f40b-49a9-a4c8-d968b53f7719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:33.417649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e8bc049c-e6f8-4415-b0d2-574292753c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:33.426674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfb8c6-f40b-49a9-a4c8-d968b53f7719 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:33.459044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 781/1556] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:33.465249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55807a18-3b97-4197-98f7-c2ca09f5e254 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.465565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55807a18-3b97-4197-98f7-c2ca09f5e254 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.466033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55807a18-3b97-4197-98f7-c2ca09f5e254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.470331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 776/1557] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:33.473344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058f47-b140-4f87-9021-4d7b101d7f04 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.473845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058f47-b140-4f87-9021-4d7b101d7f04 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.474359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058f47-b140-4f87-9021-4d7b101d7f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.543991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-55807a18-3b97-4197-98f7-c2ca09f5e254 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:33.546209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-55807a18-3b97-4197-98f7-c2ca09f5e254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:33.558930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d0058f47-b140-4f87-9021-4d7b101d7f04 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:33.561060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d0058f47-b140-4f87-9021-4d7b101d7f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:33.588694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 782/1558] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:33.590499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55807a18-3b97-4197-98f7-c2ca09f5e254 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:33.608540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 777/1559] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:33.618434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e561e-178b-4878-8f63-ac3a3de1806b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.618859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e561e-178b-4878-8f63-ac3a3de1806b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.619345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e561e-178b-4878-8f63-ac3a3de1806b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.634836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55807a18-3b97-4197-98f7-c2ca09f5e254 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:33.649543 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d267f33-646f-452b-9f51-2d898f1e39d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:33.652154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d267f33-646f-452b-9f51-2d898f1e39d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.654683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d267f33-646f-452b-9f51-2d898f1e39d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.654683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d267f33-646f-452b-9f51-2d898f1e39d3 admin admin] PATH_INFO: `/v3/projects/19420a9a70074dc8b867867e65e8627d/users/233cbb236b7c41c6953117df913a0fef/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.660835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-745e561e-178b-4878-8f63-ac3a3de1806b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:33.666007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-745e561e-178b-4878-8f63-ac3a3de1806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:33.688620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 778/1560] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:33.691146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea834d-5c74-4238-a575-7dc1d3dcd161 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.691617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea834d-5c74-4238-a575-7dc1d3dcd161 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.692003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea834d-5c74-4238-a575-7dc1d3dcd161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.712823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d267f33-646f-452b-9f51-2d898f1e39d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19420a9a70074dc8b867867e65e8627d, user_id=233cbb236b7c41c6953117df913a0fef, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:33.715037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-9d267f33-646f-452b-9f51-2d898f1e39d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 19:02:33.715037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jun 21 19:02:33.715037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: Jun 21 19:02:33.721160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d267f33-646f-452b-9f51-2d898f1e39d3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:33.730350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-33ea834d-5c74-4238-a575-7dc1d3dcd161 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:33.732078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-33ea834d-5c74-4238-a575-7dc1d3dcd161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:33.743719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 779/1561] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:33.757264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e61478-f452-43d8-aa63-bb1d2193e80c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.757264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e61478-f452-43d8-aa63-bb1d2193e80c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.757264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e61478-f452-43d8-aa63-bb1d2193e80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.757883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 783/1562] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:33 2022] PUT /identity/v3/projects/19420a9a70074dc8b867867e65e8627d/users/233cbb236b7c41c6953117df913a0fef/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:33.757883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea902db8-741e-41c8-94e4-8329a804f4f3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.757883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea902db8-741e-41c8-94e4-8329a804f4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.758781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea902db8-741e-41c8-94e4-8329a804f4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.768487 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a2e61478-f452-43d8-aa63-bb1d2193e80c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:33.770429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a2e61478-f452-43d8-aa63-bb1d2193e80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:33.807391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:33.817675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ea902db8-741e-41c8-94e4-8329a804f4f3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:33.820081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ea902db8-741e-41c8-94e4-8329a804f4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:33.910074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 780/1563] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:33.911716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e61478-f452-43d8-aa63-bb1d2193e80c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:33.926465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e61478-f452-43d8-aa63-bb1d2193e80c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:33.932393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b1633-c592-40c4-8309-c42acdcae75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:33.933298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1633-c592-40c4-8309-c42acdcae75d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.933695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1633-c592-40c4-8309-c42acdcae75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.934071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1633-c592-40c4-8309-c42acdcae75d admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.935625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b1633-c592-40c4-8309-c42acdcae75d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:02:33.936050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b1633-c592-40c4-8309-c42acdcae75d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:33.937958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b1633-c592-40c4-8309-c42acdcae75d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:33.970127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 781/1564] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:02:33.970127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb22f2-0d40-4a4a-8fa5-912d060b0bbe None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.970127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb22f2-0d40-4a4a-8fa5-912d060b0bbe None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.970127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb22f2-0d40-4a4a-8fa5-912d060b0bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.985653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 784/1565] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:33.987389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785636c-2f6c-4e3d-a38e-77dc07eebcf8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:33.987775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785636c-2f6c-4e3d-a38e-77dc07eebcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:33.987939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785636c-2f6c-4e3d-a38e-77dc07eebcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:33.992289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9ebb22f2-0d40-4a4a-8fa5-912d060b0bbe None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:33.994363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9ebb22f2-0d40-4a4a-8fa5-912d060b0bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:34.003814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 782/1566] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:34.005951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7265282-ecb8-44c5-87ec-f1dfb1cad0fd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.006344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7265282-ecb8-44c5-87ec-f1dfb1cad0fd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.006782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7265282-ecb8-44c5-87ec-f1dfb1cad0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.026768 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3785636c-2f6c-4e3d-a38e-77dc07eebcf8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:34.028928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3785636c-2f6c-4e3d-a38e-77dc07eebcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:34.055702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 785/1567] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:34.055702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d00a-230a-483d-9df5-6877fa0e80a1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.055702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d00a-230a-483d-9df5-6877fa0e80a1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.055702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d00a-230a-483d-9df5-6877fa0e80a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.058270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e7265282-ecb8-44c5-87ec-f1dfb1cad0fd None None] MFA Rules not processed for user `233cbb236b7c41c6953117df913a0fef`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:34.060560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e7265282-ecb8-44c5-87ec-f1dfb1cad0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:34.076943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1238d00a-230a-483d-9df5-6877fa0e80a1 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:34.078236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1238d00a-230a-483d-9df5-6877fa0e80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:34.086850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 786/1568] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:34.087724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d00a-230a-483d-9df5-6877fa0e80a1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.100876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d00a-230a-483d-9df5-6877fa0e80a1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.106892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dd54d-f172-42ea-a332-193a81f8d107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.107433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dd54d-f172-42ea-a332-193a81f8d107 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.107607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dd54d-f172-42ea-a332-193a81f8d107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.107936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dd54d-f172-42ea-a332-193a81f8d107 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.108679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dd54d-f172-42ea-a332-193a81f8d107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:02:34.108986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dd54d-f172-42ea-a332-193a81f8d107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.110821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dd54d-f172-42ea-a332-193a81f8d107 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.125697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 787/1569] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:02:34.126557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dd54d-f172-42ea-a332-193a81f8d107 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.140991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dd54d-f172-42ea-a332-193a81f8d107 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.148589 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda130e-a2ba-467e-b409-78bb3e0f0251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.149716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda130e-a2ba-467e-b409-78bb3e0f0251 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.150213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda130e-a2ba-467e-b409-78bb3e0f0251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.150692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda130e-a2ba-467e-b409-78bb3e0f0251 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.151917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda130e-a2ba-467e-b409-78bb3e0f0251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.153763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-bbda130e-a2ba-467e-b409-78bb3e0f0251 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 19:02:34.153763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jun 21 19:02:34.153763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: Jun 21 19:02:34.155997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda130e-a2ba-467e-b409-78bb3e0f0251 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.175409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 783/1570] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:34.177046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7265282-ecb8-44c5-87ec-f1dfb1cad0fd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.181785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:34.188254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7265282-ecb8-44c5-87ec-f1dfb1cad0fd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.193612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907409-5b91-480c-952b-8b3ce08e44b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.194499 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907409-5b91-480c-952b-8b3ce08e44b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.194783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907409-5b91-480c-952b-8b3ce08e44b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.195057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907409-5b91-480c-952b-8b3ce08e44b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.195790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61907409-5b91-480c-952b-8b3ce08e44b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:02:34.196096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61907409-5b91-480c-952b-8b3ce08e44b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.197930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61907409-5b91-480c-952b-8b3ce08e44b9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.201412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 788/1571] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.201997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f034ea8-9d28-4eb1-93dc-f91762444c45 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.202351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f034ea8-9d28-4eb1-93dc-f91762444c45 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.202700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f034ea8-9d28-4eb1-93dc-f91762444c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.214132 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 784/1572] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:02:34.215062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907409-5b91-480c-952b-8b3ce08e44b9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.225958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2f034ea8-9d28-4eb1-93dc-f91762444c45 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:34.227897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2f034ea8-9d28-4eb1-93dc-f91762444c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:34.236772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907409-5b91-480c-952b-8b3ce08e44b9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.308317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee089-c42b-4e85-84ed-d06ca6cd0942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.309116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee089-c42b-4e85-84ed-d06ca6cd0942 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.309572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee089-c42b-4e85-84ed-d06ca6cd0942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.309922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee089-c42b-4e85-84ed-d06ca6cd0942 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.310946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bee089-c42b-4e85-84ed-d06ca6cd0942 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:02:34.311330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bee089-c42b-4e85-84ed-d06ca6cd0942 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.313191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bee089-c42b-4e85-84ed-d06ca6cd0942 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.334652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 789/1573] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:34.334652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b110c93-2518-402c-a968-92b17ec7e90d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.334652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b110c93-2518-402c-a968-92b17ec7e90d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.334652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b110c93-2518-402c-a968-92b17ec7e90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.335369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 785/1574] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:02:34.335369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee089-c42b-4e85-84ed-d06ca6cd0942 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.343296 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:34.350441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee089-c42b-4e85-84ed-d06ca6cd0942 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.357439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2ebdc-81c8-4c8d-a2ca-7fef030b5f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.358290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2ebdc-81c8-4c8d-a2ca-7fef030b5f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.358725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2ebdc-81c8-4c8d-a2ca-7fef030b5f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.359052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2ebdc-81c8-4c8d-a2ca-7fef030b5f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.359906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2ebdc-81c8-4c8d-a2ca-7fef030b5f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.361940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5b110c93-2518-402c-a968-92b17ec7e90d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:34.362229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2ebdc-81c8-4c8d-a2ca-7fef030b5f69 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.363222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5b110c93-2518-402c-a968-92b17ec7e90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:34.373995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 790/1575] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:34.375265 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b110c93-2518-402c-a968-92b17ec7e90d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.387490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b110c93-2518-402c-a968-92b17ec7e90d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.393687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48533c-4040-4abf-935a-10a909af1e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.394382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48533c-4040-4abf-935a-10a909af1e94 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.395278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48533c-4040-4abf-935a-10a909af1e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.395563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48533c-4040-4abf-935a-10a909af1e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.396284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d48533c-4040-4abf-935a-10a909af1e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.398557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d48533c-4040-4abf-935a-10a909af1e94 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.400800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 786/1576] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/projects => generated 391 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.401999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2ebdc-81c8-4c8d-a2ca-7fef030b5f69 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.444366 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 791/1577] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.445738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48533c-4040-4abf-935a-10a909af1e94 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.497520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2ebdc-81c8-4c8d-a2ca-7fef030b5f69 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.544726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48533c-4040-4abf-935a-10a909af1e94 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.550143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a7ff6-bcbc-4961-8262-9826af562c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.550864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a7ff6-bcbc-4961-8262-9826af562c89 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.551157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a7ff6-bcbc-4961-8262-9826af562c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.551438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a7ff6-bcbc-4961-8262-9826af562c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.552158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a7ff6-bcbc-4961-8262-9826af562c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.554182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a7ff6-bcbc-4961-8262-9826af562c89 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.568455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b5e16-59c4-4e48-8794-4c905bf0a76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.569460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b5e16-59c4-4e48-8794-4c905bf0a76c admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.569460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b5e16-59c4-4e48-8794-4c905bf0a76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.569845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b5e16-59c4-4e48-8794-4c905bf0a76c admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.570569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b5e16-59c4-4e48-8794-4c905bf0a76c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.571689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-2a9b5e16-59c4-4e48-8794-4c905bf0a76c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 19:02:34.571689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 21 19:02:34.571689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: Jun 21 19:02:34.572931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b5e16-59c4-4e48-8794-4c905bf0a76c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.590119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 792/1578] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.591874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a7ff6-bcbc-4961-8262-9826af562c89 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.598787 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 787/1579] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.599719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b5e16-59c4-4e48-8794-4c905bf0a76c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.610707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b5e16-59c4-4e48-8794-4c905bf0a76c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.674343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:34.678590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89a589-27a3-473b-9c28-7ece05a0dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.679267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89a589-27a3-473b-9c28-7ece05a0dd70 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.679553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89a589-27a3-473b-9c28-7ece05a0dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.679881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89a589-27a3-473b-9c28-7ece05a0dd70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.680648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89a589-27a3-473b-9c28-7ece05a0dd70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.682871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89a589-27a3-473b-9c28-7ece05a0dd70 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.690097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a7ff6-bcbc-4961-8262-9826af562c89 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.697775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd06f7-dcff-4137-8e85-b89da5287fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.698473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd06f7-dcff-4137-8e85-b89da5287fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.698761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd06f7-dcff-4137-8e85-b89da5287fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.699046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd06f7-dcff-4137-8e85-b89da5287fcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.699822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd06f7-dcff-4137-8e85-b89da5287fcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:02:34.700192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd06f7-dcff-4137-8e85-b89da5287fcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.701923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd06f7-dcff-4137-8e85-b89da5287fcf admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.716035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 788/1580] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.717477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89a589-27a3-473b-9c28-7ece05a0dd70 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.730580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 793/1581] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:02:34.730580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd06f7-dcff-4137-8e85-b89da5287fcf admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.731026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:34.748585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd06f7-dcff-4137-8e85-b89da5287fcf admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.796419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89a589-27a3-473b-9c28-7ece05a0dd70 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.801750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d170d-d936-40f4-8065-b4f17d7a0ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.802612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d170d-d936-40f4-8065-b4f17d7a0ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.802899 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d170d-d936-40f4-8065-b4f17d7a0ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.803187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d170d-d936-40f4-8065-b4f17d7a0ee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.803978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d170d-d936-40f4-8065-b4f17d7a0ee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:02:34.804252 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d170d-d936-40f4-8065-b4f17d7a0ee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.805961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d170d-d936-40f4-8065-b4f17d7a0ee5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.828192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 789/1582] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:02:34.828192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d170d-d936-40f4-8065-b4f17d7a0ee5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.828490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18392bd-7304-4494-b5e7-b1382b53ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.828490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18392bd-7304-4494-b5e7-b1382b53ef05 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.828490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18392bd-7304-4494-b5e7-b1382b53ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.828490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18392bd-7304-4494-b5e7-b1382b53ef05 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.829124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18392bd-7304-4494-b5e7-b1382b53ef05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.833479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18392bd-7304-4494-b5e7-b1382b53ef05 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.842133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d170d-d936-40f4-8065-b4f17d7a0ee5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.848775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd2a09-d414-477e-aafa-0b1ecc042e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.850094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd2a09-d414-477e-aafa-0b1ecc042e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.850473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd2a09-d414-477e-aafa-0b1ecc042e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.850856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd2a09-d414-477e-aafa-0b1ecc042e9b admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.851811 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd2a09-d414-477e-aafa-0b1ecc042e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.854707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd2a09-d414-477e-aafa-0b1ecc042e9b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.861262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 794/1583] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.866380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18392bd-7304-4494-b5e7-b1382b53ef05 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.875769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18392bd-7304-4494-b5e7-b1382b53ef05 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.881137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 790/1584] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/users => generated 297 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.881580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79392829-1816-4486-8574-1f2798935795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.882335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79392829-1816-4486-8574-1f2798935795 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.882506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd2a09-d414-477e-aafa-0b1ecc042e9b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.882679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79392829-1816-4486-8574-1f2798935795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.882898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79392829-1816-4486-8574-1f2798935795 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.883602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79392829-1816-4486-8574-1f2798935795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.885653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79392829-1816-4486-8574-1f2798935795 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.901046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd2a09-d414-477e-aafa-0b1ecc042e9b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.909473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 795/1585] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/users => generated 286 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.909799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf08935-841a-47e3-8332-2119391725b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.910487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79392829-1816-4486-8574-1f2798935795 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.910938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf08935-841a-47e3-8332-2119391725b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.911450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf08935-841a-47e3-8332-2119391725b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.911868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf08935-841a-47e3-8332-2119391725b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.913684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf08935-841a-47e3-8332-2119391725b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.916797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf08935-841a-47e3-8332-2119391725b7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.922038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79392829-1816-4486-8574-1f2798935795 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.928168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc047a2-e991-4756-92bd-2f61fbc59036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.929139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc047a2-e991-4756-92bd-2f61fbc59036 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.929488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc047a2-e991-4756-92bd-2f61fbc59036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.929810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc047a2-e991-4756-92bd-2f61fbc59036 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.930268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 791/1586] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:34.930767 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc047a2-e991-4756-92bd-2f61fbc59036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.932895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc047a2-e991-4756-92bd-2f61fbc59036 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.934129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf08935-841a-47e3-8332-2119391725b7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.947066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf08935-841a-47e3-8332-2119391725b7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.952945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b7f1b-77f1-465f-af29-e2abca6a58a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.953712 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b7f1b-77f1-465f-af29-e2abca6a58a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.954000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b7f1b-77f1-465f-af29-e2abca6a58a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.954284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b7f1b-77f1-465f-af29-e2abca6a58a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.954992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b7f1b-77f1-465f-af29-e2abca6a58a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.956937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b7f1b-77f1-465f-af29-e2abca6a58a3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:34.967248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 796/1587] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/users => generated 289 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:34.968911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc047a2-e991-4756-92bd-2f61fbc59036 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:34.981399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc047a2-e991-4756-92bd-2f61fbc59036 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:34.986670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929aaf9-7a76-467c-940d-849ff62199e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:34.987545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929aaf9-7a76-467c-940d-849ff62199e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:34.987942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929aaf9-7a76-467c-940d-849ff62199e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:34.988141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929aaf9-7a76-467c-940d-849ff62199e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:34.988946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929aaf9-7a76-467c-940d-849ff62199e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:34.991405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929aaf9-7a76-467c-940d-849ff62199e3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.002511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 792/1588] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:35.004417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b7f1b-77f1-465f-af29-e2abca6a58a3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.022283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 797/1589] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:34 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:35.026276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929aaf9-7a76-467c-940d-849ff62199e3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.119613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929aaf9-7a76-467c-940d-849ff62199e3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.125433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f96688-fc90-45e5-b3b7-38e18539fe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.126379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f96688-fc90-45e5-b3b7-38e18539fe85 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.126899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f96688-fc90-45e5-b3b7-38e18539fe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.127374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f96688-fc90-45e5-b3b7-38e18539fe85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.128338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f96688-fc90-45e5-b3b7-38e18539fe85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.130905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f96688-fc90-45e5-b3b7-38e18539fe85 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.135675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b7f1b-77f1-465f-af29-e2abca6a58a3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.145464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fff779-c554-444d-b431-dca958503be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.146548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff779-c554-444d-b431-dca958503be3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.146937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff779-c554-444d-b431-dca958503be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.147361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff779-c554-444d-b431-dca958503be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.153677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 798/1590] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.153951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fff779-c554-444d-b431-dca958503be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.154117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f96688-fc90-45e5-b3b7-38e18539fe85 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.154302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fff779-c554-444d-b431-dca958503be3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.167444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f96688-fc90-45e5-b3b7-38e18539fe85 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.173143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd3dc8-5aed-4fdd-b332-290ac6a0921b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.173700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd3dc8-5aed-4fdd-b332-290ac6a0921b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.173988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd3dc8-5aed-4fdd-b332-290ac6a0921b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.174277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd3dc8-5aed-4fdd-b332-290ac6a0921b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.175029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd3dc8-5aed-4fdd-b332-290ac6a0921b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.177386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 793/1591] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.177587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd3dc8-5aed-4fdd-b332-290ac6a0921b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.179515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fff779-c554-444d-b431-dca958503be3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.189987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 799/1592] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.190621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd3dc8-5aed-4fdd-b332-290ac6a0921b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.202044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:35.205268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fff779-c554-444d-b431-dca958503be3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.207149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd3dc8-5aed-4fdd-b332-290ac6a0921b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.213626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0267e60-4756-47a3-9f6a-2f1c0e4ccd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.214482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde8526-57a5-46ab-a9fe-e8f0ca0de92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.214702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0267e60-4756-47a3-9f6a-2f1c0e4ccd4f admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.215102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0267e60-4756-47a3-9f6a-2f1c0e4ccd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.215568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0267e60-4756-47a3-9f6a-2f1c0e4ccd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.216363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde8526-57a5-46ab-a9fe-e8f0ca0de92b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.216738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde8526-57a5-46ab-a9fe-e8f0ca0de92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.216944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0267e60-4756-47a3-9f6a-2f1c0e4ccd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.217246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde8526-57a5-46ab-a9fe-e8f0ca0de92b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.218095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde8526-57a5-46ab-a9fe-e8f0ca0de92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.219652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0267e60-4756-47a3-9f6a-2f1c0e4ccd4f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.219936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde8526-57a5-46ab-a9fe-e8f0ca0de92b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.238056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 800/1593] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.240329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde8526-57a5-46ab-a9fe-e8f0ca0de92b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.245784 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 794/1594] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.247227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0267e60-4756-47a3-9f6a-2f1c0e4ccd4f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.264079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0267e60-4756-47a3-9f6a-2f1c0e4ccd4f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.264271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde8526-57a5-46ab-a9fe-e8f0ca0de92b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.271361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8172ff-fb69-4c8e-99bc-58604834afa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.271567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43e849-8043-49e8-8525-a4e9399b2601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.272411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8172ff-fb69-4c8e-99bc-58604834afa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.272552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43e849-8043-49e8-8525-a4e9399b2601 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.272889 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8172ff-fb69-4c8e-99bc-58604834afa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.273008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43e849-8043-49e8-8525-a4e9399b2601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.273429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8172ff-fb69-4c8e-99bc-58604834afa6 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.273558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43e849-8043-49e8-8525-a4e9399b2601 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.274592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8172ff-fb69-4c8e-99bc-58604834afa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.274733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43e849-8043-49e8-8525-a4e9399b2601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.278221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43e849-8043-49e8-8525-a4e9399b2601 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.278377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8172ff-fb69-4c8e-99bc-58604834afa6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.313180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 801/1595] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:35 2022] POST /identity/v3/users => generated 283 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:35.313521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 795/1596] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:35 2022] POST /identity/v3/users => generated 290 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:35.318144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8172ff-fb69-4c8e-99bc-58604834afa6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.322744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43e849-8043-49e8-8525-a4e9399b2601 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.333073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8172ff-fb69-4c8e-99bc-58604834afa6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.341983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b0e47-f4da-45fc-a344-4949d810cabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.342614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43e849-8043-49e8-8525-a4e9399b2601 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.342940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b0e47-f4da-45fc-a344-4949d810cabb admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.343231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b0e47-f4da-45fc-a344-4949d810cabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.343517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b0e47-f4da-45fc-a344-4949d810cabb admin admin] PATH_INFO: `/v3/projects/3f3338b6f51449f4a93aa94581d33db4/users/8a0c2372fbce4eed90ec598b111ef5b8/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.352383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7cbf-6dc3-4c80-a122-47d5669e5c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.353640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7cbf-6dc3-4c80-a122-47d5669e5c99 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.354306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7cbf-6dc3-4c80-a122-47d5669e5c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.354867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7cbf-6dc3-4c80-a122-47d5669e5c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.356374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae7cbf-6dc3-4c80-a122-47d5669e5c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.359345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae7cbf-6dc3-4c80-a122-47d5669e5c99 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.372394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:35.376463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b0e47-f4da-45fc-a344-4949d810cabb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3338b6f51449f4a93aa94581d33db4, user_id=8a0c2372fbce4eed90ec598b111ef5b8, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.378704 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 802/1597] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.382362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b0e47-f4da-45fc-a344-4949d810cabb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.385172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7cbf-6dc3-4c80-a122-47d5669e5c99 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.402845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7cbf-6dc3-4c80-a122-47d5669e5c99 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.409148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 796/1598] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:35 2022] PUT /identity/v3/projects/3f3338b6f51449f4a93aa94581d33db4/users/8a0c2372fbce4eed90ec598b111ef5b8/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:35.410916 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b0e47-f4da-45fc-a344-4949d810cabb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.510408 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b0e47-f4da-45fc-a344-4949d810cabb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.512079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6ea50-d8e4-41e7-95b8-2af3e6a5f7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.513293 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6ea50-d8e4-41e7-95b8-2af3e6a5f7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.513806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6ea50-d8e4-41e7-95b8-2af3e6a5f7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.514386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6ea50-d8e4-41e7-95b8-2af3e6a5f7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.515732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6ea50-d8e4-41e7-95b8-2af3e6a5f7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.517540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c9680-b8de-464a-9ce1-298ca6518f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.518398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c9680-b8de-464a-9ce1-298ca6518f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.518863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c9680-b8de-464a-9ce1-298ca6518f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.518985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6ea50-d8e4-41e7-95b8-2af3e6a5f7b0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.519408 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c9680-b8de-464a-9ce1-298ca6518f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.520874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c9680-b8de-464a-9ce1-298ca6518f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.522960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c9680-b8de-464a-9ce1-298ca6518f74 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.535115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 797/1599] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.539195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c9680-b8de-464a-9ce1-298ca6518f74 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.544447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 803/1600] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.544447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6ea50-d8e4-41e7-95b8-2af3e6a5f7b0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.550410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c9680-b8de-464a-9ce1-298ca6518f74 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.557449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09dc88-9a3a-41df-8e4f-6ba883c9bc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.558642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09dc88-9a3a-41df-8e4f-6ba883c9bc47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.558961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09dc88-9a3a-41df-8e4f-6ba883c9bc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.559559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09dc88-9a3a-41df-8e4f-6ba883c9bc47 admin admin] PATH_INFO: `/v3/projects/0e052c1c726e48ea98ee7b21c3cd55d7/users/b6ce650c0ae6494ab895644347bbb5b7/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.560741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6ea50-d8e4-41e7-95b8-2af3e6a5f7b0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.569962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5165b-6553-4574-b7dc-ccb18c722f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.571168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5165b-6553-4574-b7dc-ccb18c722f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.572271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5165b-6553-4574-b7dc-ccb18c722f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.572271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5165b-6553-4574-b7dc-ccb18c722f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.573698 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5165b-6553-4574-b7dc-ccb18c722f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.576764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5165b-6553-4574-b7dc-ccb18c722f26 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.579561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09dc88-9a3a-41df-8e4f-6ba883c9bc47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e052c1c726e48ea98ee7b21c3cd55d7, user_id=b6ce650c0ae6494ab895644347bbb5b7, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.582984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09dc88-9a3a-41df-8e4f-6ba883c9bc47 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.597630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 798/1601] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:35 2022] PUT /identity/v3/projects/0e052c1c726e48ea98ee7b21c3cd55d7/users/b6ce650c0ae6494ab895644347bbb5b7/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:35.602725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 804/1602] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.602952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09dc88-9a3a-41df-8e4f-6ba883c9bc47 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.603126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5165b-6553-4574-b7dc-ccb18c722f26 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.617515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5165b-6553-4574-b7dc-ccb18c722f26 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.678075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09dc88-9a3a-41df-8e4f-6ba883c9bc47 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.682953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85720e9-f20e-4c43-966f-810ef82ca494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.683632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85720e9-f20e-4c43-966f-810ef82ca494 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.683925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85720e9-f20e-4c43-966f-810ef82ca494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.684210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85720e9-f20e-4c43-966f-810ef82ca494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.684950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85720e9-f20e-4c43-966f-810ef82ca494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.686629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85720e9-f20e-4c43-966f-810ef82ca494 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.700053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 799/1603] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.700053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85720e9-f20e-4c43-966f-810ef82ca494 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.713346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85720e9-f20e-4c43-966f-810ef82ca494 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.717821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12c47d-4821-4a18-b71e-e3fb507647b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.718698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ed114-090f-4ac0-9705-e966cfe10456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.719433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ed114-090f-4ac0-9705-e966cfe10456 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.719571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12c47d-4821-4a18-b71e-e3fb507647b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.719828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ed114-090f-4ac0-9705-e966cfe10456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.720071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12c47d-4821-4a18-b71e-e3fb507647b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.720177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ed114-090f-4ac0-9705-e966cfe10456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.720575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12c47d-4821-4a18-b71e-e3fb507647b0 admin admin] PATH_INFO: `/v3/projects/f1af5239143346e8b995f4176d78ca4d/users/eeafe22d22f648d6aefcc6a308d4fc0d/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.721563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ed114-090f-4ac0-9705-e966cfe10456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.724562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ed114-090f-4ac0-9705-e966cfe10456 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.743373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 800/1604] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.744741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01e1e-69a4-4d3a-8dbb-bd83a8f2ed02 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.745233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01e1e-69a4-4d3a-8dbb-bd83a8f2ed02 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.745838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01e1e-69a4-4d3a-8dbb-bd83a8f2ed02 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.748668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 801/1605] 10.222.0.70 () {54 vars in 961 bytes} [Tue Jun 21 19:02:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:02:35.749735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca01e1e-69a4-4d3a-8dbb-bd83a8f2ed02 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.758231 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12c47d-4821-4a18-b71e-e3fb507647b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1af5239143346e8b995f4176d78ca4d, user_id=eeafe22d22f648d6aefcc6a308d4fc0d, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.763051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12c47d-4821-4a18-b71e-e3fb507647b0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.770727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca01e1e-69a4-4d3a-8dbb-bd83a8f2ed02 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.776872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbef6aa-402c-4860-bf55-b5ccffe48053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.779531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbef6aa-402c-4860-bf55-b5ccffe48053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.779940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbef6aa-402c-4860-bf55-b5ccffe48053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.780281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbef6aa-402c-4860-bf55-b5ccffe48053 admin admin] PATH_INFO: `/v3/projects/b6bbb7a5373e46f69acab406d82d87f3/users/2b973f8f63eb49f19895a43313f2bc55/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.781913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 805/1606] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:35 2022] PUT /identity/v3/projects/f1af5239143346e8b995f4176d78ca4d/users/eeafe22d22f648d6aefcc6a308d4fc0d/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:35.786940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12c47d-4821-4a18-b71e-e3fb507647b0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.801209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbef6aa-402c-4860-bf55-b5ccffe48053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bbb7a5373e46f69acab406d82d87f3, user_id=2b973f8f63eb49f19895a43313f2bc55, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.806120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbef6aa-402c-4860-bf55-b5ccffe48053 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.826423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 802/1607] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:35 2022] PUT /identity/v3/projects/b6bbb7a5373e46f69acab406d82d87f3/users/2b973f8f63eb49f19895a43313f2bc55/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:35.828516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbef6aa-402c-4860-bf55-b5ccffe48053 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:35.840638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:35.866450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:35.887641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12c47d-4821-4a18-b71e-e3fb507647b0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.924240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbef6aa-402c-4860-bf55-b5ccffe48053 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:35.932383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f226e-c61b-4d4c-b1ee-ce45abcb2914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.933495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f226e-c61b-4d4c-b1ee-ce45abcb2914 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.933938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f226e-c61b-4d4c-b1ee-ce45abcb2914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.934375 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f226e-c61b-4d4c-b1ee-ce45abcb2914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.935612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f226e-c61b-4d4c-b1ee-ce45abcb2914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:35.938397 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f226e-c61b-4d4c-b1ee-ce45abcb2914 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:35.957591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 803/1608] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:35.960098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a3029-1fff-44ea-9798-f7e28d2b6491 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.960857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a3029-1fff-44ea-9798-f7e28d2b6491 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.961360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a3029-1fff-44ea-9798-f7e28d2b6491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.975387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cb2c3-5370-404f-b761-87bbf6de4d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:35.976321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cb2c3-5370-404f-b761-87bbf6de4d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:35.976609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cb2c3-5370-404f-b761-87bbf6de4d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:35.976889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cb2c3-5370-404f-b761-87bbf6de4d64 admin admin] PATH_INFO: `/v3/projects/bd0c3a0d5898414384ce780f51951d01/users/1347dcdfd097453490b144bf22cbea76/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:35.983880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d04a3029-1fff-44ea-9798-f7e28d2b6491 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:35.985440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d04a3029-1fff-44ea-9798-f7e28d2b6491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:35.995986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cb2c3-5370-404f-b761-87bbf6de4d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0c3a0d5898414384ce780f51951d01, user_id=1347dcdfd097453490b144bf22cbea76, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:36.001068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 804/1609] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:36.001413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cb2c3-5370-404f-b761-87bbf6de4d64 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:36.001582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a3029-1fff-44ea-9798-f7e28d2b6491 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:36.017140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a3029-1fff-44ea-9798-f7e28d2b6491 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:36.017863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 806/1610] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:35 2022] PUT /identity/v3/projects/bd0c3a0d5898414384ce780f51951d01/users/1347dcdfd097453490b144bf22cbea76/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:36.019256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cb2c3-5370-404f-b761-87bbf6de4d64 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:36.117022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cb2c3-5370-404f-b761-87bbf6de4d64 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:36.122809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c1ec7-a82b-4c33-9dda-7861e2ca8782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:36.123956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1ec7-a82b-4c33-9dda-7861e2ca8782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.124418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1ec7-a82b-4c33-9dda-7861e2ca8782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.124768 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1ec7-a82b-4c33-9dda-7861e2ca8782 admin admin] PATH_INFO: `/v3/projects/3f3338b6f51449f4a93aa94581d33db4/users/8a0c2372fbce4eed90ec598b111ef5b8/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.127347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c1ec7-a82b-4c33-9dda-7861e2ca8782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3338b6f51449f4a93aa94581d33db4, user_id=8a0c2372fbce4eed90ec598b111ef5b8, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:36.131055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c1ec7-a82b-4c33-9dda-7861e2ca8782 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:36.131289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a73f3-692c-4973-afee-7ba5d15baafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:36.132320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a73f3-692c-4973-afee-7ba5d15baafa admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.132843 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a73f3-692c-4973-afee-7ba5d15baafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.133409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a73f3-692c-4973-afee-7ba5d15baafa admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.134871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a73f3-692c-4973-afee-7ba5d15baafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:36.137893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a73f3-692c-4973-afee-7ba5d15baafa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:36.144571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 807/1611] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:36 2022] PUT /identity/v3/projects/3f3338b6f51449f4a93aa94581d33db4/users/8a0c2372fbce4eed90ec598b111ef5b8/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:36.146334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4045659-820d-4e9b-a2d2-3ec9f87db8ad None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.146705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4045659-820d-4e9b-a2d2-3ec9f87db8ad None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.146890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4045659-820d-4e9b-a2d2-3ec9f87db8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.160246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 805/1612] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:36.163018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ce6b-8147-4473-a409-fb4c1ddfbc3b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.163358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ce6b-8147-4473-a409-fb4c1ddfbc3b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.163856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ce6b-8147-4473-a409-fb4c1ddfbc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.170024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b4045659-820d-4e9b-a2d2-3ec9f87db8ad None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:36.171265 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b4045659-820d-4e9b-a2d2-3ec9f87db8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:36.194037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d986ce6b-8147-4473-a409-fb4c1ddfbc3b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:36.195676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d986ce6b-8147-4473-a409-fb4c1ddfbc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:36.248793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:36.277622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 808/1613] 10.222.0.70 () {58 vars in 1058 bytes} [Tue Jun 21 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:36.279672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3587-59b2-4ba0-a691-f526a3c45ff2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.280071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3587-59b2-4ba0-a691-f526a3c45ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.280532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3587-59b2-4ba0-a691-f526a3c45ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.302309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 806/1614] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:36.303742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986ce6b-8147-4473-a409-fb4c1ddfbc3b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:36.308631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f22e3587-59b2-4ba0-a691-f526a3c45ff2 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:36.310725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f22e3587-59b2-4ba0-a691-f526a3c45ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:36.320247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986ce6b-8147-4473-a409-fb4c1ddfbc3b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:36.323505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 809/1615] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:36.328440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab75c-3f0b-4dec-81d4-f83f1cb9ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:36.329468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e3587-59b2-4ba0-a691-f526a3c45ff2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:36.329926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab75c-3f0b-4dec-81d4-f83f1cb9ea18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.330837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab75c-3f0b-4dec-81d4-f83f1cb9ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.330837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab75c-3f0b-4dec-81d4-f83f1cb9ea18 admin admin] PATH_INFO: `/v3/projects/5c339aeed9774c598bf07c6c10c0c551/users/cc1b5b7df1194b7bac68ba381ceb8f60/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.355582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ab75c-3f0b-4dec-81d4-f83f1cb9ea18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c339aeed9774c598bf07c6c10c0c551, user_id=cc1b5b7df1194b7bac68ba381ceb8f60, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:36.358731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ab75c-3f0b-4dec-81d4-f83f1cb9ea18 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:36.376433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 807/1616] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:36 2022] PUT /identity/v3/projects/5c339aeed9774c598bf07c6c10c0c551/users/cc1b5b7df1194b7bac68ba381ceb8f60/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:36.378091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa38a2-4181-4e88-8248-fed17bbb1ca7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.378495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa38a2-4181-4e88-8248-fed17bbb1ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.378955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa38a2-4181-4e88-8248-fed17bbb1ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.413480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-72aa38a2-4181-4e88-8248-fed17bbb1ca7 None None] MFA Rules not processed for user `1347dcdfd097453490b144bf22cbea76`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:36.414990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e3587-59b2-4ba0-a691-f526a3c45ff2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:36.415410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-72aa38a2-4181-4e88-8248-fed17bbb1ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:36.449387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:36.492938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2015ae-212a-4617-adf9-6b2b2f054b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:36.494135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2015ae-212a-4617-adf9-6b2b2f054b45 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.494658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2015ae-212a-4617-adf9-6b2b2f054b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.495163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2015ae-212a-4617-adf9-6b2b2f054b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.522797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 808/1617] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:36.524225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5be11-400c-443e-abf1-36030006b837 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.524570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5be11-400c-443e-abf1-36030006b837 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.524847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5be11-400c-443e-abf1-36030006b837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.555757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-36c5be11-400c-443e-abf1-36030006b837 None None] MFA Rules not processed for user `8a0c2372fbce4eed90ec598b111ef5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:36.557002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-36c5be11-400c-443e-abf1-36030006b837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:36.569587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2015ae-212a-4617-adf9-6b2b2f054b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:36.571241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2015ae-212a-4617-adf9-6b2b2f054b45 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:36.585118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 810/1618] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:36.587324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2015ae-212a-4617-adf9-6b2b2f054b45 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:36.603877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2015ae-212a-4617-adf9-6b2b2f054b45 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:36.612556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4499c-fd20-4283-816b-868bd3aa9dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:36.614721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4499c-fd20-4283-816b-868bd3aa9dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.615235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4499c-fd20-4283-816b-868bd3aa9dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.615750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4499c-fd20-4283-816b-868bd3aa9dd9 admin admin] PATH_INFO: `/v3/projects/d22f07bc3f5641d19b56a56257d0c951/users/a7ee05285e124183a43f5757fd94ec40/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.635598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4499c-fd20-4283-816b-868bd3aa9dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22f07bc3f5641d19b56a56257d0c951, user_id=a7ee05285e124183a43f5757fd94ec40, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:36.638934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4499c-fd20-4283-816b-868bd3aa9dd9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:36.646511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 809/1619] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:36.648045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b74db6-4e5d-4d47-970c-b19c9f2b08b0 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.648347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b74db6-4e5d-4d47-970c-b19c9f2b08b0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.648609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b74db6-4e5d-4d47-970c-b19c9f2b08b0 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.653445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 810/1620] 10.222.0.70 () {54 vars in 912 bytes} [Tue Jun 21 19:02:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:02:36.654713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b74db6-4e5d-4d47-970c-b19c9f2b08b0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:36.663090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 811/1621] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:36 2022] PUT /identity/v3/projects/d22f07bc3f5641d19b56a56257d0c951/users/a7ee05285e124183a43f5757fd94ec40/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:36.669141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4499c-fd20-4283-816b-868bd3aa9dd9 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:36.669141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-08e4499c-fd20-4283-816b-868bd3aa9dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:36.700551 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:36.741279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4499c-fd20-4283-816b-868bd3aa9dd9 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:36.747286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b74db6-4e5d-4d47-970c-b19c9f2b08b0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:36.748847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7b096-163c-4eb5-997b-4d054143d109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:36.749930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b096-163c-4eb5-997b-4d054143d109 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.750335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b096-163c-4eb5-997b-4d054143d109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.750703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b096-163c-4eb5-997b-4d054143d109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.751741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3ad14-5082-4afd-b4dc-0b9fc3e9eeee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:36.752543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3ad14-5082-4afd-b4dc-0b9fc3e9eeee service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.752827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3ad14-5082-4afd-b4dc-0b9fc3e9eeee service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.753123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3ad14-5082-4afd-b4dc-0b9fc3e9eeee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.830302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:36.850002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3ad14-5082-4afd-b4dc-0b9fc3e9eeee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:36.851957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3ad14-5082-4afd-b4dc-0b9fc3e9eeee service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:36.869011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 811/1622] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:36.870700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc7e4-c520-474f-97d9-06b0c0ee6c4e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.871038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc7e4-c520-474f-97d9-06b0c0ee6c4e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.871322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc7e4-c520-474f-97d9-06b0c0ee6c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.890790 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7b096-163c-4eb5-997b-4d054143d109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:36.893897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7b096-163c-4eb5-997b-4d054143d109 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:36.916852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 812/1623] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:36.918505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7b096-163c-4eb5-997b-4d054143d109 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:36.926893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-508dc7e4-c520-474f-97d9-06b0c0ee6c4e None None] MFA Rules not processed for user `cc1b5b7df1194b7bac68ba381ceb8f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:36.929322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-508dc7e4-c520-474f-97d9-06b0c0ee6c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:36.948279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7b096-163c-4eb5-997b-4d054143d109 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:36.961104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d2d1e-c31b-4a6e-93af-268f34b3d84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:36.962631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d2d1e-c31b-4a6e-93af-268f34b3d84f admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:36.963279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d2d1e-c31b-4a6e-93af-268f34b3d84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:36.963805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d2d1e-c31b-4a6e-93af-268f34b3d84f admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:36.965442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d2d1e-c31b-4a6e-93af-268f34b3d84f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:36.969785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d2d1e-c31b-4a6e-93af-268f34b3d84f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:37.026352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 813/1624] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:36 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:37.030199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dab94-4927-4128-a25f-14c4d47d1f3e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.030971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dab94-4927-4128-a25f-14c4d47d1f3e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.031612 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dab94-4927-4128-a25f-14c4d47d1f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.062316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 812/1625] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:37.064064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347ff81-bdd9-4ef6-ab36-45506e52f049 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.064595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347ff81-bdd9-4ef6-ab36-45506e52f049 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.064979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347ff81-bdd9-4ef6-ab36-45506e52f049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.081015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d72dab94-4927-4128-a25f-14c4d47d1f3e None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:37.084863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d72dab94-4927-4128-a25f-14c4d47d1f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:37.091238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b347ff81-bdd9-4ef6-ab36-45506e52f049 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:37.092696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b347ff81-bdd9-4ef6-ab36-45506e52f049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:37.224341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 813/1626] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:37.224341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347ff81-bdd9-4ef6-ab36-45506e52f049 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:37.281634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 814/1627] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:37.282407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dab94-4927-4128-a25f-14c4d47d1f3e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:37.300753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dab94-4927-4128-a25f-14c4d47d1f3e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:37.305983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd008e5-a90c-4d89-9644-b260a59216e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:37.306680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd008e5-a90c-4d89-9644-b260a59216e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.307097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd008e5-a90c-4d89-9644-b260a59216e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.307385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd008e5-a90c-4d89-9644-b260a59216e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.308191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd008e5-a90c-4d89-9644-b260a59216e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:37.310615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd008e5-a90c-4d89-9644-b260a59216e6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:37.323679 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347ff81-bdd9-4ef6-ab36-45506e52f049 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:37.329647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f1a5a-afcd-4837-9c02-6ce7eaed53b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:37.330858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1a5a-afcd-4837-9c02-6ce7eaed53b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.331483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1a5a-afcd-4837-9c02-6ce7eaed53b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.332688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1a5a-afcd-4837-9c02-6ce7eaed53b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.353425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 815/1628] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:37 2022] POST /identity/v3/users => generated 292 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:37.354984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd008e5-a90c-4d89-9644-b260a59216e6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:37.372689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd008e5-a90c-4d89-9644-b260a59216e6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:37.383743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb1b0c-c220-4651-a420-b529773f397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:37.384520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:37.384915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb1b0c-c220-4651-a420-b529773f397e admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.385628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb1b0c-c220-4651-a420-b529773f397e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.385974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb1b0c-c220-4651-a420-b529773f397e admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.387315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb1b0c-c220-4651-a420-b529773f397e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:37.391841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb1b0c-c220-4651-a420-b529773f397e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:37.421348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f1a5a-afcd-4837-9c02-6ce7eaed53b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:37.423054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f1a5a-afcd-4837-9c02-6ce7eaed53b0 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:37.436685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 814/1629] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:37.438328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f1a5a-afcd-4837-9c02-6ce7eaed53b0 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:37.469919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 816/1630] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:02:37 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:37.471644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb1b0c-c220-4651-a420-b529773f397e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:37.481247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:37.574219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f1a5a-afcd-4837-9c02-6ce7eaed53b0 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:37.636857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb1b0c-c220-4651-a420-b529773f397e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:37.662118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde3e91-f1c0-48a1-9fcd-c54a1ea9b3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:37.667915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde3e91-f1c0-48a1-9fcd-c54a1ea9b3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.669409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde3e91-f1c0-48a1-9fcd-c54a1ea9b3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.669807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924ecd0-d48f-48e6-a55a-f96f2f6470c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:37.671177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924ecd0-d48f-48e6-a55a-f96f2f6470c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.671842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924ecd0-d48f-48e6-a55a-f96f2f6470c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.672429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924ecd0-d48f-48e6-a55a-f96f2f6470c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.673503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde3e91-f1c0-48a1-9fcd-c54a1ea9b3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.807947 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924ecd0-d48f-48e6-a55a-f96f2f6470c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:37.810126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924ecd0-d48f-48e6-a55a-f96f2f6470c4 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:37.824450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 815/1631] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:37.825673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924ecd0-d48f-48e6-a55a-f96f2f6470c4 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:37.836378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924ecd0-d48f-48e6-a55a-f96f2f6470c4 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:37.843824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97f084-aa78-46a8-bb0c-6ed44333b0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:37.844782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f084-aa78-46a8-bb0c-6ed44333b0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.845506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f084-aa78-46a8-bb0c-6ed44333b0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.845942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f084-aa78-46a8-bb0c-6ed44333b0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.846865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97f084-aa78-46a8-bb0c-6ed44333b0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:37.849055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97f084-aa78-46a8-bb0c-6ed44333b0e7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:37.857952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde3e91-f1c0-48a1-9fcd-c54a1ea9b3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:37.860749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 816/1632] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:37.866308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde3e91-f1c0-48a1-9fcd-c54a1ea9b3e6 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:37.866516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97f084-aa78-46a8-bb0c-6ed44333b0e7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:37.879448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 817/1633] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:37.880439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde3e91-f1c0-48a1-9fcd-c54a1ea9b3e6 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:37.893381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97f084-aa78-46a8-bb0c-6ed44333b0e7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:37.893785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde3e91-f1c0-48a1-9fcd-c54a1ea9b3e6 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:37.898559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab57c4-0ed5-4532-b5fe-ed01959f40ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:37.899246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab57c4-0ed5-4532-b5fe-ed01959f40ed service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.899435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab57c4-0ed5-4532-b5fe-ed01959f40ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.899778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab57c4-0ed5-4532-b5fe-ed01959f40ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.913151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67243c4-8850-42dc-bf40-bcd5b299db54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:37.915376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67243c4-8850-42dc-bf40-bcd5b299db54 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:37.916860 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67243c4-8850-42dc-bf40-bcd5b299db54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:37.918137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67243c4-8850-42dc-bf40-bcd5b299db54 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:37.919796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67243c4-8850-42dc-bf40-bcd5b299db54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:37.927030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67243c4-8850-42dc-bf40-bcd5b299db54 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:37.985057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab57c4-0ed5-4532-b5fe-ed01959f40ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:37.989630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab57c4-0ed5-4532-b5fe-ed01959f40ed service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:37.990238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 817/1634] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:02:37 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:02:38.005207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67243c4-8850-42dc-bf40-bcd5b299db54 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:38.007182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 818/1635] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:38.008622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab57c4-0ed5-4532-b5fe-ed01959f40ed service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:38.023541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab57c4-0ed5-4532-b5fe-ed01959f40ed service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:38.031567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9076-5747-43ab-8521-a88e306170c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:38.032862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9076-5747-43ab-8521-a88e306170c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:38.032862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9076-5747-43ab-8521-a88e306170c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:38.033117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9076-5747-43ab-8521-a88e306170c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:38.034227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e9076-5747-43ab-8521-a88e306170c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:38.039113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e9076-5747-43ab-8521-a88e306170c3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:38.046117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67243c4-8850-42dc-bf40-bcd5b299db54 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:38.058532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 819/1636] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:38.065006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a64d2-1454-4133-8ee1-fb98913d0f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:38.065265 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9076-5747-43ab-8521-a88e306170c3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:38.065364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a64d2-1454-4133-8ee1-fb98913d0f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:38.065825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a64d2-1454-4133-8ee1-fb98913d0f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:38.066612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a64d2-1454-4133-8ee1-fb98913d0f7a admin admin] PATH_INFO: `/v3/projects/8cbe803de7d341fe9c62dbe9549c207e/users/eda0bd8bde2a4dddbf8c23173490e2bb/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:38.072417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:38.080711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9076-5747-43ab-8521-a88e306170c3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:38.087861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb927169-a911-4db3-ab64-60c412a95645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:38.089487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb927169-a911-4db3-ab64-60c412a95645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:38.089979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb927169-a911-4db3-ab64-60c412a95645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:38.090361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb927169-a911-4db3-ab64-60c412a95645 admin admin] PATH_INFO: `/v3/projects/8f4cf23fdf7849fbb84e534df9ef08e6/users/8a03fd0c0a614045848c7cd882e5ddc0/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:38.117819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a64d2-1454-4133-8ee1-fb98913d0f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbe803de7d341fe9c62dbe9549c207e, user_id=eda0bd8bde2a4dddbf8c23173490e2bb, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:38.121308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb927169-a911-4db3-ab64-60c412a95645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f4cf23fdf7849fbb84e534df9ef08e6, user_id=8a03fd0c0a614045848c7cd882e5ddc0, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:38.124383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a64d2-1454-4133-8ee1-fb98913d0f7a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:38.124604 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb927169-a911-4db3-ab64-60c412a95645 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:38.154878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 820/1637] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:38 2022] PUT /identity/v3/projects/8f4cf23fdf7849fbb84e534df9ef08e6/users/8a03fd0c0a614045848c7cd882e5ddc0/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:38.159823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb927169-a911-4db3-ab64-60c412a95645 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:38.169558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 818/1638] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:37 2022] PUT /identity/v3/projects/8cbe803de7d341fe9c62dbe9549c207e/users/eda0bd8bde2a4dddbf8c23173490e2bb/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:38.171794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a64d2-1454-4133-8ee1-fb98913d0f7a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:38.189588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:38.340781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a64d2-1454-4133-8ee1-fb98913d0f7a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:38.349069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfe4f8-b2ee-4e16-9837-4bce2e25559a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:38.349990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfe4f8-b2ee-4e16-9837-4bce2e25559a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:38.350382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfe4f8-b2ee-4e16-9837-4bce2e25559a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:38.350661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfe4f8-b2ee-4e16-9837-4bce2e25559a admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:38.352083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfe4f8-b2ee-4e16-9837-4bce2e25559a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:38.354167 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfe4f8-b2ee-4e16-9837-4bce2e25559a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:38.363493 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb927169-a911-4db3-ab64-60c412a95645 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:38.369135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 819/1639] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:38.372327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfe4f8-b2ee-4e16-9837-4bce2e25559a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:38.385145 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9b450-f704-4ac6-8ef5-e5ea6c872b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:38.385846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9b450-f704-4ac6-8ef5-e5ea6c872b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:38.386178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9b450-f704-4ac6-8ef5-e5ea6c872b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:38.387985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9b450-f704-4ac6-8ef5-e5ea6c872b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:38.390149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9b450-f704-4ac6-8ef5-e5ea6c872b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:38.391930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfe4f8-b2ee-4e16-9837-4bce2e25559a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:38.395934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9b450-f704-4ac6-8ef5-e5ea6c872b71 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:38.399249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da127f-f5cd-480a-a2c4-979ba2fe4628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:38.400482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da127f-f5cd-480a-a2c4-979ba2fe4628 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:38.400974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da127f-f5cd-480a-a2c4-979ba2fe4628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:38.401655 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da127f-f5cd-480a-a2c4-979ba2fe4628 admin admin] PATH_INFO: `/v3/projects/8cbe803de7d341fe9c62dbe9549c207e/users/eda0bd8bde2a4dddbf8c23173490e2bb/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:38.405318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da127f-f5cd-480a-a2c4-979ba2fe4628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbe803de7d341fe9c62dbe9549c207e, user_id=eda0bd8bde2a4dddbf8c23173490e2bb, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:38.424589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da127f-f5cd-480a-a2c4-979ba2fe4628 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:38.439391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 821/1640] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:02:38.440322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9b450-f704-4ac6-8ef5-e5ea6c872b71 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:38.468634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9b450-f704-4ac6-8ef5-e5ea6c872b71 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:38.503553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 820/1641] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:38 2022] PUT /identity/v3/projects/8cbe803de7d341fe9c62dbe9549c207e/users/eda0bd8bde2a4dddbf8c23173490e2bb/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:38.526959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda70cd3-bf11-4bcb-90f8-db069faef840 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:38.527551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda70cd3-bf11-4bcb-90f8-db069faef840 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:38.527922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda70cd3-bf11-4bcb-90f8-db069faef840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:38.594028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25118df-b4fe-4f3f-8322-20bb1672064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:38.594914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-eda70cd3-bf11-4bcb-90f8-db069faef840 None None] MFA Rules not processed for user `eda0bd8bde2a4dddbf8c23173490e2bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:38.595367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25118df-b4fe-4f3f-8322-20bb1672064b admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:38.595814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25118df-b4fe-4f3f-8322-20bb1672064b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:38.596266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25118df-b4fe-4f3f-8322-20bb1672064b admin admin] PATH_INFO: `/v3/projects/8f4cf23fdf7849fbb84e534df9ef08e6/users/8a03fd0c0a614045848c7cd882e5ddc0/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:38.596454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-eda70cd3-bf11-4bcb-90f8-db069faef840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:38.606129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25118df-b4fe-4f3f-8322-20bb1672064b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f4cf23fdf7849fbb84e534df9ef08e6, user_id=8a03fd0c0a614045848c7cd882e5ddc0, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:38.619218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25118df-b4fe-4f3f-8322-20bb1672064b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:38.641632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 822/1642] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:02:38 2022] PUT /identity/v3/projects/8f4cf23fdf7849fbb84e534df9ef08e6/users/8a03fd0c0a614045848c7cd882e5ddc0/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:02:38.657304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8d123-4df0-4324-bdda-18a38ae8b3e3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:38.657526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8d123-4df0-4324-bdda-18a38ae8b3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:38.662662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8d123-4df0-4324-bdda-18a38ae8b3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:38.762618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-def8d123-4df0-4324-bdda-18a38ae8b3e3 None None] MFA Rules not processed for user `8a03fd0c0a614045848c7cd882e5ddc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:38.764313 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-def8d123-4df0-4324-bdda-18a38ae8b3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:38.807580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 821/1643] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:38 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:38.821329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda70cd3-bf11-4bcb-90f8-db069faef840 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:38.921021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 823/1644] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:38 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:38.944050 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8d123-4df0-4324-bdda-18a38ae8b3e3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:38.982497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda70cd3-bf11-4bcb-90f8-db069faef840 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:38.995973 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:39.009341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:39.084204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8d123-4df0-4324-bdda-18a38ae8b3e3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:39.088260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e58a9-5526-4452-afe4-1b452ecd2587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:39.088933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e58a9-5526-4452-afe4-1b452ecd2587 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:39.089253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e58a9-5526-4452-afe4-1b452ecd2587 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:39.089535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e58a9-5526-4452-afe4-1b452ecd2587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:39.102851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e58a9-5526-4452-afe4-1b452ecd2587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:39.105580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e58a9-5526-4452-afe4-1b452ecd2587 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:39.124245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 824/1645] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:39.148770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71ca90-1092-45e3-80d6-407c91442432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:39.150098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71ca90-1092-45e3-80d6-407c91442432 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:39.150549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71ca90-1092-45e3-80d6-407c91442432 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:39.151054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71ca90-1092-45e3-80d6-407c91442432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:39.250253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71ca90-1092-45e3-80d6-407c91442432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:39.252156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71ca90-1092-45e3-80d6-407c91442432 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:39.265875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 822/1646] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:39.440138 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e58a9-5526-4452-afe4-1b452ecd2587 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:39.457498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e58a9-5526-4452-afe4-1b452ecd2587 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:39.469076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c71fa4-35da-4e7a-8640-166af5a2916a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:39.470049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c71fa4-35da-4e7a-8640-166af5a2916a service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:39.471352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c71fa4-35da-4e7a-8640-166af5a2916a service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:39.474015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c71fa4-35da-4e7a-8640-166af5a2916a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:39.487788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71ca90-1092-45e3-80d6-407c91442432 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:39.508234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71ca90-1092-45e3-80d6-407c91442432 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:39.520605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbebbd-dce1-4482-9894-918674781da8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:39.521218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbebbd-dce1-4482-9894-918674781da8 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:39.521603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbebbd-dce1-4482-9894-918674781da8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:39.521928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbebbd-dce1-4482-9894-918674781da8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:39.602757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c71fa4-35da-4e7a-8640-166af5a2916a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:39.604378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c71fa4-35da-4e7a-8640-166af5a2916a service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:39.618508 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 825/1647] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:39.704670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbebbd-dce1-4482-9894-918674781da8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:39.706664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbebbd-dce1-4482-9894-918674781da8 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:39.722299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 823/1648] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:40.462494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54e28e-07fc-4c0c-81c8-c491c78ab927 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:40.463364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54e28e-07fc-4c0c-81c8-c491c78ab927 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:40.463795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54e28e-07fc-4c0c-81c8-c491c78ab927 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:40.466095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 826/1649] 10.222.0.70 () {54 vars in 961 bytes} [Tue Jun 21 19:02:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:02:40.475994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848b9a-af3e-417d-b382-7e19c6addaad None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:40.476554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848b9a-af3e-417d-b382-7e19c6addaad None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:40.476554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848b9a-af3e-417d-b382-7e19c6addaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:40.530568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4c848b9a-af3e-417d-b382-7e19c6addaad None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:40.532160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4c848b9a-af3e-417d-b382-7e19c6addaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:40.698338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:40.746550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 824/1650] 10.222.0.70 () {58 vars in 1058 bytes} [Tue Jun 21 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:40.751887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405dff1-febf-4eaa-b60b-f5fe382abf6f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:40.755756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405dff1-febf-4eaa-b60b-f5fe382abf6f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:40.755756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405dff1-febf-4eaa-b60b-f5fe382abf6f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:40.755756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 827/1651] 10.222.0.70 () {54 vars in 912 bytes} [Tue Jun 21 19:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:02:40.761864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12067b5f-7ef0-4b26-9ab1-2de72008ba51 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:40.762710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12067b5f-7ef0-4b26-9ab1-2de72008ba51 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:40.763798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12067b5f-7ef0-4b26-9ab1-2de72008ba51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:40.810165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-12067b5f-7ef0-4b26-9ab1-2de72008ba51 None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:40.813204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-12067b5f-7ef0-4b26-9ab1-2de72008ba51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:40.843289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 825/1652] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:02:41.192264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405dff1-febf-4eaa-b60b-f5fe382abf6f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:41.214179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405dff1-febf-4eaa-b60b-f5fe382abf6f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:41.218477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fff806-a06f-4f3a-a077-6db58f10eb42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:41.219126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fff806-a06f-4f3a-a077-6db58f10eb42 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:41.219418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fff806-a06f-4f3a-a077-6db58f10eb42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:41.219720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fff806-a06f-4f3a-a077-6db58f10eb42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:41.244382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fff806-a06f-4f3a-a077-6db58f10eb42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:41.248948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fff806-a06f-4f3a-a077-6db58f10eb42 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:41.295192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 828/1653] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:50.791181 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=8 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:51.277475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=49 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:52.133464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b634e-76b7-4707-bb92-30751d3c4a82 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:52.134751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b634e-76b7-4707-bb92-30751d3c4a82 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:52.135239 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b634e-76b7-4707-bb92-30751d3c4a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:52.201814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-411b634e-76b7-4707-bb92-30751d3c4a82 None None] MFA Rules not processed for user `b6ce650c0ae6494ab895644347bbb5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:52.203526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-411b634e-76b7-4707-bb92-30751d3c4a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:52.353532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 826/1654] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:52.376655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fff806-a06f-4f3a-a077-6db58f10eb42 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:52.395742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fff806-a06f-4f3a-a077-6db58f10eb42 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:52.411969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76d17f-a86a-4b56-bc8a-8fa7f61ed2f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:52.414697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76d17f-a86a-4b56-bc8a-8fa7f61ed2f6 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:52.415855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76d17f-a86a-4b56-bc8a-8fa7f61ed2f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:52.417194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76d17f-a86a-4b56-bc8a-8fa7f61ed2f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:52.461013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76d17f-a86a-4b56-bc8a-8fa7f61ed2f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:52.464010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76d17f-a86a-4b56-bc8a-8fa7f61ed2f6 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:52.513288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 829/1655] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:02:52.822781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13657506-ba83-4e9c-a08a-83ef6964f08e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:52.823639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13657506-ba83-4e9c-a08a-83ef6964f08e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:52.824142 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13657506-ba83-4e9c-a08a-83ef6964f08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:52.906200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-13657506-ba83-4e9c-a08a-83ef6964f08e None None] MFA Rules not processed for user `2b973f8f63eb49f19895a43313f2bc55`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:52.907530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-13657506-ba83-4e9c-a08a-83ef6964f08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:53.057672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 827/1656] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:53.075507 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76d17f-a86a-4b56-bc8a-8fa7f61ed2f6 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:53.130061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76d17f-a86a-4b56-bc8a-8fa7f61ed2f6 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:53.157881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4518-fba5-4212-aec3-15def692eff8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:53.159906 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4518-fba5-4212-aec3-15def692eff8 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:53.159906 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4518-fba5-4212-aec3-15def692eff8 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:53.163362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4518-fba5-4212-aec3-15def692eff8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:53.208593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a4518-fba5-4212-aec3-15def692eff8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:53.212267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a4518-fba5-4212-aec3-15def692eff8 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:53.236509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 830/1657] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:55.085598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b226563-5753-43fb-9b74-31b6b3aaa723 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:55.086545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b226563-5753-43fb-9b74-31b6b3aaa723 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:55.086988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b226563-5753-43fb-9b74-31b6b3aaa723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:55.181210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9b226563-5753-43fb-9b74-31b6b3aaa723 None None] MFA Rules not processed for user `eeafe22d22f648d6aefcc6a308d4fc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:55.183005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9b226563-5753-43fb-9b74-31b6b3aaa723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:55.503089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 828/1658] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:55.520677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e522e3-0329-47c1-a345-d87549982dcc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:55.521654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e522e3-0329-47c1-a345-d87549982dcc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:55.522546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e522e3-0329-47c1-a345-d87549982dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:55.628783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-30e522e3-0329-47c1-a345-d87549982dcc None None] MFA Rules not processed for user `eeafe22d22f648d6aefcc6a308d4fc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:55.638498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-30e522e3-0329-47c1-a345-d87549982dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:55.666133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 831/1659] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:55.674247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b226563-5753-43fb-9b74-31b6b3aaa723 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:55.711092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b226563-5753-43fb-9b74-31b6b3aaa723 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:55.741372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363de457-6205-4cf6-95ef-3d170215f5fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:55.742660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363de457-6205-4cf6-95ef-3d170215f5fa service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:55.742968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363de457-6205-4cf6-95ef-3d170215f5fa service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:55.743243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363de457-6205-4cf6-95ef-3d170215f5fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:55.761129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363de457-6205-4cf6-95ef-3d170215f5fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:55.766333 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363de457-6205-4cf6-95ef-3d170215f5fa service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:55.770615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:02:55.798963 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 829/1660] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:56.890747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55778ee6-bd0d-4dcd-a64d-88d9dd1d03dd None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:56.891655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55778ee6-bd0d-4dcd-a64d-88d9dd1d03dd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:56.891808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55778ee6-bd0d-4dcd-a64d-88d9dd1d03dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:56.984742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-55778ee6-bd0d-4dcd-a64d-88d9dd1d03dd None None] MFA Rules not processed for user `a7ee05285e124183a43f5757fd94ec40`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:56.990429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-55778ee6-bd0d-4dcd-a64d-88d9dd1d03dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:57.171120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 832/1661] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:57.196546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a033579-cad3-4a8d-8fc0-1046e886f8d1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:57.197354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a033579-cad3-4a8d-8fc0-1046e886f8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:57.197354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a033579-cad3-4a8d-8fc0-1046e886f8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:57.240230 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5a033579-cad3-4a8d-8fc0-1046e886f8d1 None None] MFA Rules not processed for user `a7ee05285e124183a43f5757fd94ec40`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:57.243564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5a033579-cad3-4a8d-8fc0-1046e886f8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:57.265449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 830/1662] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:57.273881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55778ee6-bd0d-4dcd-a64d-88d9dd1d03dd None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:57.276638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f950d-f15b-473d-813d-3dc3b47a3467 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:57.277654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f950d-f15b-473d-813d-3dc3b47a3467 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:57.278583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f950d-f15b-473d-813d-3dc3b47a3467 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:57.282430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 831/1663] 10.222.0.70 () {54 vars in 912 bytes} [Tue Jun 21 19:02:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:02:57.291833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b0620-b246-4d92-afaf-dc63cfc24bb6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:57.292605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b0620-b246-4d92-afaf-dc63cfc24bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:57.293301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b0620-b246-4d92-afaf-dc63cfc24bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:57.308441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55778ee6-bd0d-4dcd-a64d-88d9dd1d03dd None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:57.333041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e09d1-6629-4d65-ad97-4d9d8d687c4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:57.334165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e09d1-6629-4d65-ad97-4d9d8d687c4c service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:57.334670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e09d1-6629-4d65-ad97-4d9d8d687c4c service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:57.335171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e09d1-6629-4d65-ad97-4d9d8d687c4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:57.352685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e09d1-6629-4d65-ad97-4d9d8d687c4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:57.354561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e09d1-6629-4d65-ad97-4d9d8d687c4c service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:57.397569 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 833/1664] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:57.411150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-076b0620-b246-4d92-afaf-dc63cfc24bb6 None None] MFA Rules not processed for user `808b530d24b3487b9a20ee74c131502e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:57.412468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-076b0620-b246-4d92-afaf-dc63cfc24bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:57.694114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 832/1665] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:02:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:57.735651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e09d1-6629-4d65-ad97-4d9d8d687c4c service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:57.755045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e09d1-6629-4d65-ad97-4d9d8d687c4c service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:57.764990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c59295-f224-40f2-8c25-8a14bceec101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:57.765712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c59295-f224-40f2-8c25-8a14bceec101 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:57.766005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c59295-f224-40f2-8c25-8a14bceec101 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:57.766786 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c59295-f224-40f2-8c25-8a14bceec101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:57.813894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c59295-f224-40f2-8c25-8a14bceec101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:57.821285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c59295-f224-40f2-8c25-8a14bceec101 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:57.855794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 834/1666] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:02:57.856162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea88c8-1f9f-4da0-a3de-a1ae54469e90 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:57.856776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea88c8-1f9f-4da0-a3de-a1ae54469e90 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:57.857012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea88c8-1f9f-4da0-a3de-a1ae54469e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:57.902919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d4ea88c8-1f9f-4da0-a3de-a1ae54469e90 None None] MFA Rules not processed for user `2b973f8f63eb49f19895a43313f2bc55`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:57.905516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d4ea88c8-1f9f-4da0-a3de-a1ae54469e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:57.943518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 833/1667] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:02:58.222350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa0230-f71a-420a-9d45-45e8cf643680 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:58.223961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa0230-f71a-420a-9d45-45e8cf643680 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:58.224978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa0230-f71a-420a-9d45-45e8cf643680 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:58.228402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 835/1668] 10.222.0.70 () {54 vars in 912 bytes} [Tue Jun 21 19:02:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:02:58.233933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252983-b84e-4071-a6a3-ae20b7120ece None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:58.234904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252983-b84e-4071-a6a3-ae20b7120ece None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:58.235292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252983-b84e-4071-a6a3-ae20b7120ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:58.259496 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fb252983-b84e-4071-a6a3-ae20b7120ece None None] MFA Rules not processed for user `808b530d24b3487b9a20ee74c131502e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:58.261361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fb252983-b84e-4071-a6a3-ae20b7120ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:58.270686 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 834/1669] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:02:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:58.314918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa0230-f71a-420a-9d45-45e8cf643680 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:58.345375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa0230-f71a-420a-9d45-45e8cf643680 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:58.348283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab270c-c1b7-4011-8469-bc6972a92f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:58.349347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab270c-c1b7-4011-8469-bc6972a92f0d service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:58.349793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab270c-c1b7-4011-8469-bc6972a92f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:58.350113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab270c-c1b7-4011-8469-bc6972a92f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:58.360245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab270c-c1b7-4011-8469-bc6972a92f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:58.361765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab270c-c1b7-4011-8469-bc6972a92f0d service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:58.379578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 836/1670] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:58.979288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9e5c-3198-4804-9164-10fe7a4c9c35 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:58.980369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9e5c-3198-4804-9164-10fe7a4c9c35 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:58.980827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9e5c-3198-4804-9164-10fe7a4c9c35 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:58.984289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 835/1671] 10.222.0.70 () {54 vars in 960 bytes} [Tue Jun 21 19:02:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:02:58.993882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda28615-75e9-4059-a989-dc9b8052a019 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:58.994182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda28615-75e9-4059-a989-dc9b8052a019 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:58.994513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda28615-75e9-4059-a989-dc9b8052a019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:59.016457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-dda28615-75e9-4059-a989-dc9b8052a019 None None] MFA Rules not processed for user `f0e3fc619dc74a0591d8ad14369ae673`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:59.017758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dda28615-75e9-4059-a989-dc9b8052a019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:59.142802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 837/1672] 10.222.0.70 () {58 vars in 1057 bytes} [Tue Jun 21 19:02:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:02:59.170801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b9e5c-3198-4804-9164-10fe7a4c9c35 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:02:59.189822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7616d0-9056-42f1-9f3c-43438443fde4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:59.190099 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7616d0-9056-42f1-9f3c-43438443fde4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:59.190386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7616d0-9056-42f1-9f3c-43438443fde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:59.230775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b9e5c-3198-4804-9164-10fe7a4c9c35 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:02:59.231063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8a7616d0-9056-42f1-9f3c-43438443fde4 None None] MFA Rules not processed for user `b6ce650c0ae6494ab895644347bbb5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:02:59.232258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8a7616d0-9056-42f1-9f3c-43438443fde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:59.256779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 838/1673] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:02:59.271342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67bd23-f558-4c83-95a0-4bf5ebc60458 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:02:59.273645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67bd23-f558-4c83-95a0-4bf5ebc60458 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:59.275131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67bd23-f558-4c83-95a0-4bf5ebc60458 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:59.275939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67bd23-f558-4c83-95a0-4bf5ebc60458 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:59.289293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8b67bd23-f558-4c83-95a0-4bf5ebc60458 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:02:59.327628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67bd23-f558-4c83-95a0-4bf5ebc60458 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:02:59.340643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67bd23-f558-4c83-95a0-4bf5ebc60458 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:02:59.387177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 836/1674] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:02:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:02:59.930551 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc32c3d-a5a8-4dcb-83e7-01e0ebbd1965 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:02:59.931032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc32c3d-a5a8-4dcb-83e7-01e0ebbd1965 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:02:59.931449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc32c3d-a5a8-4dcb-83e7-01e0ebbd1965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:02:59.971767 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:00.006377 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3cc32c3d-a5a8-4dcb-83e7-01e0ebbd1965 None None] MFA Rules not processed for user `eeafe22d22f648d6aefcc6a308d4fc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:00.016333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3cc32c3d-a5a8-4dcb-83e7-01e0ebbd1965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:00.053243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 839/1675] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:02:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:02.116313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67bd23-f558-4c83-95a0-4bf5ebc60458 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:02.307656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67bd23-f558-4c83-95a0-4bf5ebc60458 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:02.317521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:02.320227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:02.323252 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:02.323998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] PATH_INFO: `/v3/users/233cbb236b7c41c6953117df913a0fef` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:02.331323 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=233cbb236b7c41c6953117df913a0fef)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:02.338348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:02.556069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '233cbb236b7c41c6953117df913a0fef'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:02.617514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:02.627641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '233cbb236b7c41c6953117df913a0fef'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:02.688483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '233cbb236b7c41c6953117df913a0fef'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:02.715978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '233cbb236b7c41c6953117df913a0fef'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:02.716885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '233cbb236b7c41c6953117df913a0fef'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:02.732560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 837/1676] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:02 2022] DELETE /identity/v3/users/233cbb236b7c41c6953117df913a0fef => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:02.738850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1339-b890-4cc4-b34b-bde0ec6dabb6 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:02.740019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1339-b890-4cc4-b34b-bde0ec6dabb6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:02.740488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1339-b890-4cc4-b34b-bde0ec6dabb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:02.784714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b14f1339-b890-4cc4-b34b-bde0ec6dabb6 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:02.786852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b14f1339-b890-4cc4-b34b-bde0ec6dabb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:03.030893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 840/1677] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:03.033697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:03.035435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:03.211578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee0aa4-85c2-4de6-b2e1-93a80f824d0e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:03.222623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a825cb-7e83-45bb-80fc-3f5a7aa904c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:03.223338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a825cb-7e83-45bb-80fc-3f5a7aa904c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:03.223570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a825cb-7e83-45bb-80fc-3f5a7aa904c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:03.224040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a825cb-7e83-45bb-80fc-3f5a7aa904c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:03.266643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a825cb-7e83-45bb-80fc-3f5a7aa904c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:03.268209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a825cb-7e83-45bb-80fc-3f5a7aa904c4 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:03.304796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 838/1678] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:03.810789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f1339-b890-4cc4-b34b-bde0ec6dabb6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:03.814574 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b14f1339-b890-4cc4-b34b-bde0ec6dabb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:03.871687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f1339-b890-4cc4-b34b-bde0ec6dabb6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:03.895361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:03.902557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:03.903508 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:03.904682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] PATH_INFO: `/v3/users/355cc6f51cc64835b059262f841c2bbc` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:03.912086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=355cc6f51cc64835b059262f841c2bbc)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:03.920618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 19:03:03.920618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jun 21 19:03:03.920618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: Jun 21 19:03:03.929902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:04.370206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '355cc6f51cc64835b059262f841c2bbc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:04.474544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '355cc6f51cc64835b059262f841c2bbc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:04.505806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '355cc6f51cc64835b059262f841c2bbc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:04.507014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '355cc6f51cc64835b059262f841c2bbc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:04.523321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '355cc6f51cc64835b059262f841c2bbc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:04.538794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 841/1679] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:03 2022] DELETE /identity/v3/users/355cc6f51cc64835b059262f841c2bbc => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:04.541637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b8082-f695-4467-954f-8135e0d78c4f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:04.542557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b8082-f695-4467-954f-8135e0d78c4f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:04.543070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b8082-f695-4467-954f-8135e0d78c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:04.703584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f43b8082-f695-4467-954f-8135e0d78c4f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:04.720719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f43b8082-f695-4467-954f-8135e0d78c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:05.069353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 839/1680] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 529 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:05.153014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:05.163300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:05.210951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b8082-f695-4467-954f-8135e0d78c4f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:05.214199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f43b8082-f695-4467-954f-8135e0d78c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:05.234501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf838f48-4fd5-4a38-afeb-3bc2ade757cb admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:05.263905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:05.277486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:05.279486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:05.280865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] PATH_INFO: `/v3/projects/19420a9a70074dc8b867867e65e8627d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:05.287597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=19420a9a70074dc8b867867e65e8627d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:05.304015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:05.492609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b8082-f695-4467-954f-8135e0d78c4f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:05.562119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19420a9a70074dc8b867867e65e8627d', 'request_id': 'req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:05.637681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19420a9a70074dc8b867867e65e8627d', 'request_id': 'req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:05.658770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:05.687870 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19420a9a70074dc8b867867e65e8627d', 'request_id': 'req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:05.709891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c7603-0198-4047-848a-246fc8cb817e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:05.710694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c7603-0198-4047-848a-246fc8cb817e service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:05.710861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c7603-0198-4047-848a-246fc8cb817e service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:05.714819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c7603-0198-4047-848a-246fc8cb817e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:05.742286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:05.825625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] The token cache is being invalidate because project 19420a9a70074dc8b867867e65e8627d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:05.841597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:05.849727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:05.865913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 842/1681] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:05 2022] DELETE /identity/v3/projects/19420a9a70074dc8b867867e65e8627d => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:05.867867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:05.889422 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:05.914153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c7603-0198-4047-848a-246fc8cb817e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:05.915685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c7603-0198-4047-848a-246fc8cb817e service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:05.944965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1d4b3-4ebb-4c46-bb1c-69ec8c17d25a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:05.948129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 840/1682] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 737 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:05.949440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c117cd5-1193-475f-a0fd-6b26895f4039 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:05.949742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c117cd5-1193-475f-a0fd-6b26895f4039 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:05.950035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c117cd5-1193-475f-a0fd-6b26895f4039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:05.967604 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:05.972581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:05.974356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:05.977737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] PATH_INFO: `/v3/users/cc1b5b7df1194b7bac68ba381ceb8f60` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:05.981221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc1b5b7df1194b7bac68ba381ceb8f60)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:05.997789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:06.041646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9c117cd5-1193-475f-a0fd-6b26895f4039 None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:06.051711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9c117cd5-1193-475f-a0fd-6b26895f4039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:06.160498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 841/1683] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:03:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:06.166388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c117cd5-1193-475f-a0fd-6b26895f4039 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:06.168804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9c117cd5-1193-475f-a0fd-6b26895f4039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:06.194508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c117cd5-1193-475f-a0fd-6b26895f4039 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:06.204425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc1b5b7df1194b7bac68ba381ceb8f60'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.226999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:06.228536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:06.229044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:06.229601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] PATH_INFO: `/v3/users/8a0c2372fbce4eed90ec598b111ef5b8` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:06.246121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0c2372fbce4eed90ec598b111ef5b8)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:06.250436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:06.287456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc1b5b7df1194b7bac68ba381ceb8f60'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.316710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc1b5b7df1194b7bac68ba381ceb8f60'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.318439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc1b5b7df1194b7bac68ba381ceb8f60'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.353532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-309ef42a-0414-4ceb-9275-28d2cf53758c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc1b5b7df1194b7bac68ba381ceb8f60'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.391272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 843/1684] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:05 2022] DELETE /identity/v3/users/cc1b5b7df1194b7bac68ba381ceb8f60 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:06.397283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461e178-c445-4974-a7b8-53bcd4e55fbf None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:06.398218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461e178-c445-4974-a7b8-53bcd4e55fbf None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:06.399041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461e178-c445-4974-a7b8-53bcd4e55fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:06.406682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0c2372fbce4eed90ec598b111ef5b8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.471324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0c2372fbce4eed90ec598b111ef5b8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.474979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5461e178-c445-4974-a7b8-53bcd4e55fbf None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:06.477649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5461e178-c445-4974-a7b8-53bcd4e55fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:06.509295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0c2372fbce4eed90ec598b111ef5b8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.550428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0c2372fbce4eed90ec598b111ef5b8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.552090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0c2372fbce4eed90ec598b111ef5b8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:06.580254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 842/1685] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:06 2022] DELETE /identity/v3/users/8a0c2372fbce4eed90ec598b111ef5b8 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:06.585169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:06.591307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:06.624723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:06.728395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 844/1686] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:06.909832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835c7ae-058c-4f8f-a22a-67bd36d16c57 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:06.944942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914a4e5-4263-43b9-9610-a531bebb7db6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:06.946496 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914a4e5-4263-43b9-9610-a531bebb7db6 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:06.947204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914a4e5-4263-43b9-9610-a531bebb7db6 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:06.947955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914a4e5-4263-43b9-9610-a531bebb7db6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:06.973747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9914a4e5-4263-43b9-9610-a531bebb7db6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:07.259614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461e178-c445-4974-a7b8-53bcd4e55fbf None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:07.262874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5461e178-c445-4974-a7b8-53bcd4e55fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:07.291970 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914a4e5-4263-43b9-9610-a531bebb7db6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:07.299106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914a4e5-4263-43b9-9610-a531bebb7db6 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:07.350144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 843/1687] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:03:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:03:07.493921 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461e178-c445-4974-a7b8-53bcd4e55fbf None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:07.608566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:07.734136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c03b10-d579-422c-a32d-df4ba420fc8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:07.735291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c03b10-d579-422c-a32d-df4ba420fc8e service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:07.735794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c03b10-d579-422c-a32d-df4ba420fc8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:07.736516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c03b10-d579-422c-a32d-df4ba420fc8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:08.048117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c03b10-d579-422c-a32d-df4ba420fc8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:08.051086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c03b10-d579-422c-a32d-df4ba420fc8e service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:08.092680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 845/1688] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 834 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:09.089989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76ec8-07e7-4f97-84a8-ba1fd981ef42 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:09.090855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76ec8-07e7-4f97-84a8-ba1fd981ef42 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:09.090855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76ec8-07e7-4f97-84a8-ba1fd981ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:09.157791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a8c76ec8-07e7-4f97-84a8-ba1fd981ef42 None None] MFA Rules not processed for user `808b530d24b3487b9a20ee74c131502e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:09.166798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a8c76ec8-07e7-4f97-84a8-ba1fd981ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:09.203501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 844/1689] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jun 21 19:03:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:09.630919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c03b10-d579-422c-a32d-df4ba420fc8e service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:09.642028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c03b10-d579-422c-a32d-df4ba420fc8e service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:09.671962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad3e49-af0a-4165-b561-7325c93a307e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:09.672995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad3e49-af0a-4165-b561-7325c93a307e service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:09.673614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad3e49-af0a-4165-b561-7325c93a307e service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:09.674170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad3e49-af0a-4165-b561-7325c93a307e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:09.735558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad3e49-af0a-4165-b561-7325c93a307e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:09.739023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad3e49-af0a-4165-b561-7325c93a307e service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:09.788667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 846/1690] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:10.073197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c76ec8-07e7-4f97-84a8-ba1fd981ef42 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:10.074786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a8c76ec8-07e7-4f97-84a8-ba1fd981ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:10.130328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c76ec8-07e7-4f97-84a8-ba1fd981ef42 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:10.151456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:10.152907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:10.152907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:10.154811 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] PATH_INFO: `/v3/projects/3f3338b6f51449f4a93aa94581d33db4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:10.159689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f3338b6f51449f4a93aa94581d33db4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:10.169857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 19:03:10.169857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 21 19:03:10.169857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: Jun 21 19:03:10.175554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:10.561994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f3338b6f51449f4a93aa94581d33db4', 'request_id': 'req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:10.598189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f3338b6f51449f4a93aa94581d33db4', 'request_id': 'req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:10.629687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f3338b6f51449f4a93aa94581d33db4', 'request_id': 'req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:10.708238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] The token cache is being invalidate because project 3f3338b6f51449f4a93aa94581d33db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:10.708869 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:10.710107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-507ea1a8-35e9-464d-a00c-f0a9c6b6240a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:10.713500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 845/1691] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:10 2022] DELETE /identity/v3/projects/3f3338b6f51449f4a93aa94581d33db4 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:10.719663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2fb00-041e-47fd-b168-6e91d5a5bbed None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:10.720657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2fb00-041e-47fd-b168-6e91d5a5bbed None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:10.721217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2fb00-041e-47fd-b168-6e91d5a5bbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:10.786651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9de2fb00-041e-47fd-b168-6e91d5a5bbed None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:10.791854 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9de2fb00-041e-47fd-b168-6e91d5a5bbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:11.182334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 847/1692] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:11.192741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d40b3-a314-4610-bd41-a7647273de7a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:11.193467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d40b3-a314-4610-bd41-a7647273de7a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:11.193980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d40b3-a314-4610-bd41-a7647273de7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:11.290877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-855d40b3-a314-4610-bd41-a7647273de7a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:11.293229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-855d40b3-a314-4610-bd41-a7647273de7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:11.339241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 846/1693] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:11.344494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2fb00-041e-47fd-b168-6e91d5a5bbed None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:11.382715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2fb00-041e-47fd-b168-6e91d5a5bbed None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:11.413402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf495-2af5-4da4-a163-7f855dfd5e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:11.416872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf495-2af5-4da4-a163-7f855dfd5e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:11.417608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf495-2af5-4da4-a163-7f855dfd5e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:11.418190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf495-2af5-4da4-a163-7f855dfd5e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:11.425419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf495-2af5-4da4-a163-7f855dfd5e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:11.426309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf495-2af5-4da4-a163-7f855dfd5e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:11.429835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf495-2af5-4da4-a163-7f855dfd5e4b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:11.526852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 848/1694] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:11.539882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d40b3-a314-4610-bd41-a7647273de7a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:11.593316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d40b3-a314-4610-bd41-a7647273de7a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:11.625488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdb7e7-eddb-4749-be49-24a481f2f94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:11.627971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdb7e7-eddb-4749-be49-24a481f2f94c admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:11.628729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdb7e7-eddb-4749-be49-24a481f2f94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:11.629405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdb7e7-eddb-4749-be49-24a481f2f94c admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:11.641490 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fdb7e7-eddb-4749-be49-24a481f2f94c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:11.646544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fdb7e7-eddb-4749-be49-24a481f2f94c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:11.705017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:11.750142 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 847/1695] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:11 2022] POST /identity/v3/projects => generated 363 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:11.756108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf495-2af5-4da4-a163-7f855dfd5e4b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:11.806778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf495-2af5-4da4-a163-7f855dfd5e4b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:11.852398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdb7e7-eddb-4749-be49-24a481f2f94c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:11.856008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-73fdb7e7-eddb-4749-be49-24a481f2f94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:12.213263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdb7e7-eddb-4749-be49-24a481f2f94c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:12.222823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:12.226066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:12.226658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:12.227163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:12.228022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] PATH_INFO: `/v3/projects/500d748635b54d98b53d1f708456ec71` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:12.236056 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=500d748635b54d98b53d1f708456ec71)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:12.247700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:12.294888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77c704-7ffa-46e8-809d-7ab431e861d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:12.295896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77c704-7ffa-46e8-809d-7ab431e861d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:12.298580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77c704-7ffa-46e8-809d-7ab431e861d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:12.299061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77c704-7ffa-46e8-809d-7ab431e861d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:12.299996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77c704-7ffa-46e8-809d-7ab431e861d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:12.303123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77c704-7ffa-46e8-809d-7ab431e861d3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:12.392141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 849/1696] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:11 2022] POST /identity/v3/users => generated 282 bytes in 637 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:12.396201 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77c704-7ffa-46e8-809d-7ab431e861d3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:12.460013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77c704-7ffa-46e8-809d-7ab431e861d3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:12.498638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '500d748635b54d98b53d1f708456ec71', 'request_id': 'req-770fe27e-4901-48ab-9313-050ce74364f0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:12.537104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '500d748635b54d98b53d1f708456ec71', 'request_id': 'req-770fe27e-4901-48ab-9313-050ce74364f0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:12.578072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '500d748635b54d98b53d1f708456ec71', 'request_id': 'req-770fe27e-4901-48ab-9313-050ce74364f0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:12.613256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] The token cache is being invalidate because project 500d748635b54d98b53d1f708456ec71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:12.617007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:12.621881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-770fe27e-4901-48ab-9313-050ce74364f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:12.625168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 848/1697] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:11 2022] DELETE /identity/v3/projects/500d748635b54d98b53d1f708456ec71 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:12.631923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1045df-9c68-4eda-b5fc-81cc3be07ee6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:12.635739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1045df-9c68-4eda-b5fc-81cc3be07ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:12.636529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1045df-9c68-4eda-b5fc-81cc3be07ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:12.686640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-eb1045df-9c68-4eda-b5fc-81cc3be07ee6 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:12.692067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-eb1045df-9c68-4eda-b5fc-81cc3be07ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:12.713197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633429c6-0ee7-49a8-9bef-30185587f4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:12.715390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633429c6-0ee7-49a8-9bef-30185587f4aa admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:12.715854 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633429c6-0ee7-49a8-9bef-30185587f4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:12.717056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633429c6-0ee7-49a8-9bef-30185587f4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:12.718008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633429c6-0ee7-49a8-9bef-30185587f4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:12.723102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633429c6-0ee7-49a8-9bef-30185587f4aa admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:12.745299 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 850/1698] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:12.747491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633429c6-0ee7-49a8-9bef-30185587f4aa admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:12.760111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-633429c6-0ee7-49a8-9bef-30185587f4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:12.844338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633429c6-0ee7-49a8-9bef-30185587f4aa admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:12.853284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 849/1699] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:12.862456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a8a33-524b-417d-86a1-1c37f3859d86 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:12.864165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a8a33-524b-417d-86a1-1c37f3859d86 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:12.864586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a8a33-524b-417d-86a1-1c37f3859d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:12.870271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afe437-f79b-4b71-8233-048770f7b3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:12.871919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afe437-f79b-4b71-8233-048770f7b3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:12.872458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afe437-f79b-4b71-8233-048770f7b3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:12.874624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afe437-f79b-4b71-8233-048770f7b3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:12.876636 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afe437-f79b-4b71-8233-048770f7b3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:12.885813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afe437-f79b-4b71-8233-048770f7b3e0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:12.906743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f66a8a33-524b-417d-86a1-1c37f3859d86 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:12.909473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f66a8a33-524b-417d-86a1-1c37f3859d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:12.935063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 850/1700] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:12.940373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a8a33-524b-417d-86a1-1c37f3859d86 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:12.961470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a8a33-524b-417d-86a1-1c37f3859d86 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:12.978338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4db00e-92d8-47d1-9a96-33e6fc60b077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:12.979716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db00e-92d8-47d1-9a96-33e6fc60b077 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:12.980127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db00e-92d8-47d1-9a96-33e6fc60b077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:12.980526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db00e-92d8-47d1-9a96-33e6fc60b077 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:12.993584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4db00e-92d8-47d1-9a96-33e6fc60b077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:12.993584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4db00e-92d8-47d1-9a96-33e6fc60b077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:12.993584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4db00e-92d8-47d1-9a96-33e6fc60b077 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:12.994285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 851/1701] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:12.994285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afe437-f79b-4b71-8233-048770f7b3e0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:13.044112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afe437-f79b-4b71-8233-048770f7b3e0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:13.047151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 851/1702] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:13.052494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4db00e-92d8-47d1-9a96-33e6fc60b077 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:13.092580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65116a-a3de-4ad4-873c-b3f75a544783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:13.099191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65116a-a3de-4ad4-873c-b3f75a544783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:13.101183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65116a-a3de-4ad4-873c-b3f75a544783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:13.103866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65116a-a3de-4ad4-873c-b3f75a544783 admin admin] PATH_INFO: `/v3/projects/6995d7b9eeab49bf9888ebe4039ad54c/users/da5614a3a21942e6a67bfb830c66139a/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:13.107284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:13.124535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4db00e-92d8-47d1-9a96-33e6fc60b077 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:13.145652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd162e8f-87c2-4669-b24e-b7e6a2a97168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:13.146548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd162e8f-87c2-4669-b24e-b7e6a2a97168 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:13.147032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd162e8f-87c2-4669-b24e-b7e6a2a97168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:13.147427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd162e8f-87c2-4669-b24e-b7e6a2a97168 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:13.148271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd162e8f-87c2-4669-b24e-b7e6a2a97168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:13.156213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd162e8f-87c2-4669-b24e-b7e6a2a97168 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:13.163832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65116a-a3de-4ad4-873c-b3f75a544783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6995d7b9eeab49bf9888ebe4039ad54c, user_id=da5614a3a21942e6a67bfb830c66139a, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:13.168387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65116a-a3de-4ad4-873c-b3f75a544783 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:13.223545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 852/1703] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:12 2022] PUT /identity/v3/projects/6995d7b9eeab49bf9888ebe4039ad54c/users/da5614a3a21942e6a67bfb830c66139a/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:13.228164 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40691e-f542-449f-96fb-83c769339d9c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:13.228909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40691e-f542-449f-96fb-83c769339d9c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:13.229879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40691e-f542-449f-96fb-83c769339d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:13.261670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 852/1704] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:13 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:13.264704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd162e8f-87c2-4669-b24e-b7e6a2a97168 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:13.343961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9b40691e-f542-449f-96fb-83c769339d9c None None] MFA Rules not processed for user `da5614a3a21942e6a67bfb830c66139a`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:13.348648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9b40691e-f542-449f-96fb-83c769339d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:13.485683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd162e8f-87c2-4669-b24e-b7e6a2a97168 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:13.499704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90149469-1127-47bc-ae47-fb99a34b671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:13.502862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90149469-1127-47bc-ae47-fb99a34b671f admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:13.503521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90149469-1127-47bc-ae47-fb99a34b671f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:13.503820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90149469-1127-47bc-ae47-fb99a34b671f admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:13.506723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90149469-1127-47bc-ae47-fb99a34b671f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:13.509573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90149469-1127-47bc-ae47-fb99a34b671f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:13.560131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 853/1705] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:13.562277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40691e-f542-449f-96fb-83c769339d9c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:13.568246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 853/1706] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:13 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:13.572541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90149469-1127-47bc-ae47-fb99a34b671f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:13.591297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40691e-f542-449f-96fb-83c769339d9c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:13.596915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90149469-1127-47bc-ae47-fb99a34b671f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:13.603055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49228af3-5dda-4d67-a077-45f81104d8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:13.604029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49228af3-5dda-4d67-a077-45f81104d8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:13.604643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49228af3-5dda-4d67-a077-45f81104d8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:13.605668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49228af3-5dda-4d67-a077-45f81104d8a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:13.607876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49228af3-5dda-4d67-a077-45f81104d8a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:13.615556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809dd47-465a-41c1-bbdc-39929c66b869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:13.615995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49228af3-5dda-4d67-a077-45f81104d8a2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:13.624772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809dd47-465a-41c1-bbdc-39929c66b869 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:13.624772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809dd47-465a-41c1-bbdc-39929c66b869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:13.624772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809dd47-465a-41c1-bbdc-39929c66b869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:13.624772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809dd47-465a-41c1-bbdc-39929c66b869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:13.625886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809dd47-465a-41c1-bbdc-39929c66b869 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:13.653273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 854/1707] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:13.654451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809dd47-465a-41c1-bbdc-39929c66b869 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:13.683420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 854/1708] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:13 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:13.684653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49228af3-5dda-4d67-a077-45f81104d8a2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:13.687315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809dd47-465a-41c1-bbdc-39929c66b869 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:13.720211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:13.966918 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e71b4-e562-4f01-919b-819df7664c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:13.967652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e71b4-e562-4f01-919b-819df7664c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:13.967950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e71b4-e562-4f01-919b-819df7664c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:13.968345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e71b4-e562-4f01-919b-819df7664c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:13.969249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e71b4-e562-4f01-919b-819df7664c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:13.975727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49228af3-5dda-4d67-a077-45f81104d8a2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:13.976455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e71b4-e562-4f01-919b-819df7664c67 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:13.997198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd4b6c-efd1-4912-8057-12fa4c6668cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:13.997950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd4b6c-efd1-4912-8057-12fa4c6668cc admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:13.998325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd4b6c-efd1-4912-8057-12fa4c6668cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:13.998608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd4b6c-efd1-4912-8057-12fa4c6668cc admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:13.999415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd4b6c-efd1-4912-8057-12fa4c6668cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:14.011223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 855/1709] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:14.011394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd4b6c-efd1-4912-8057-12fa4c6668cc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:14.015994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e71b4-e562-4f01-919b-819df7664c67 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:14.065395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e71b4-e562-4f01-919b-819df7664c67 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:14.077644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5b7bb-2ff2-4a51-9ccc-59d2c70e2c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:14.078800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5b7bb-2ff2-4a51-9ccc-59d2c70e2c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:14.079144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5b7bb-2ff2-4a51-9ccc-59d2c70e2c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:14.079477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5b7bb-2ff2-4a51-9ccc-59d2c70e2c26 admin admin] PATH_INFO: `/v3/projects/5e839f79cc74463b8fff3e45c302c5e9/users/e222c1107db143cfb25c3bf297e90e70/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:14.090402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 855/1710] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:13 2022] POST /identity/v3/users => generated 288 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:14.094595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd4b6c-efd1-4912-8057-12fa4c6668cc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:14.141527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5b7bb-2ff2-4a51-9ccc-59d2c70e2c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e839f79cc74463b8fff3e45c302c5e9, user_id=e222c1107db143cfb25c3bf297e90e70, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:14.150162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5b7bb-2ff2-4a51-9ccc-59d2c70e2c26 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:14.165518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd4b6c-efd1-4912-8057-12fa4c6668cc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:14.181603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 856/1711] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:14 2022] PUT /identity/v3/projects/5e839f79cc74463b8fff3e45c302c5e9/users/e222c1107db143cfb25c3bf297e90e70/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:14.193325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcce5d-3b8e-489c-bd5c-7a46b41d5527 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:14.193635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcce5d-3b8e-489c-bd5c-7a46b41d5527 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:14.193911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcce5d-3b8e-489c-bd5c-7a46b41d5527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:14.197228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60a37a-defc-4c34-a473-192cec1dba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:14.199638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60a37a-defc-4c34-a473-192cec1dba9f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:14.199970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60a37a-defc-4c34-a473-192cec1dba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:14.200221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60a37a-defc-4c34-a473-192cec1dba9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:14.203990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60a37a-defc-4c34-a473-192cec1dba9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:14.210783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60a37a-defc-4c34-a473-192cec1dba9f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:14.244230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 856/1712] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:14.247294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60a37a-defc-4c34-a473-192cec1dba9f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:14.255110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7bbcce5d-3b8e-489c-bd5c-7a46b41d5527 None None] MFA Rules not processed for user `e222c1107db143cfb25c3bf297e90e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:14.256673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7bbcce5d-3b8e-489c-bd5c-7a46b41d5527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:14.296313 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60a37a-defc-4c34-a473-192cec1dba9f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:14.328138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:14.477861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 857/1713] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:14.482636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcce5d-3b8e-489c-bd5c-7a46b41d5527 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:14.557033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db7f27-80d9-442f-8d2d-a17819413e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:14.558348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db7f27-80d9-442f-8d2d-a17819413e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:14.558679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db7f27-80d9-442f-8d2d-a17819413e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:14.558968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db7f27-80d9-442f-8d2d-a17819413e0a admin admin] PATH_INFO: `/v3/projects/f70c0761172f446cad9a73670a54eaec/users/0c86934c6a8249ec9f740cacc2970eb7/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:14.625225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db7f27-80d9-442f-8d2d-a17819413e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70c0761172f446cad9a73670a54eaec, user_id=0c86934c6a8249ec9f740cacc2970eb7, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:14.628290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db7f27-80d9-442f-8d2d-a17819413e0a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:14.642814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcce5d-3b8e-489c-bd5c-7a46b41d5527 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:14.652282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68f255-26c0-4dc2-a75f-3c5f0f2f9680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:14.656936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68f255-26c0-4dc2-a75f-3c5f0f2f9680 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:14.657258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68f255-26c0-4dc2-a75f-3c5f0f2f9680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:14.659301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68f255-26c0-4dc2-a75f-3c5f0f2f9680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:14.660083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68f255-26c0-4dc2-a75f-3c5f0f2f9680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:14.663184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68f255-26c0-4dc2-a75f-3c5f0f2f9680 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:14.672396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 857/1714] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:14 2022] PUT /identity/v3/projects/f70c0761172f446cad9a73670a54eaec/users/0c86934c6a8249ec9f740cacc2970eb7/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:14.675785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db7f27-80d9-442f-8d2d-a17819413e0a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:14.681604 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-26db7f27-80d9-442f-8d2d-a17819413e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:14.729039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 858/1715] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:14 2022] POST /identity/v3/projects => generated 365 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:14.735519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68f255-26c0-4dc2-a75f-3c5f0f2f9680 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:14.856676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db7f27-80d9-442f-8d2d-a17819413e0a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:14.890058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:14.913889 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68f255-26c0-4dc2-a75f-3c5f0f2f9680 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:14.928663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576497e1-daeb-408a-a4ff-8a9d232e50dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:14.929396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576497e1-daeb-408a-a4ff-8a9d232e50dc admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:14.929687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576497e1-daeb-408a-a4ff-8a9d232e50dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:14.929987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576497e1-daeb-408a-a4ff-8a9d232e50dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:14.930814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576497e1-daeb-408a-a4ff-8a9d232e50dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:14.932686 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576497e1-daeb-408a-a4ff-8a9d232e50dc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:14.968374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 859/1716] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:14.969468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576497e1-daeb-408a-a4ff-8a9d232e50dc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:14.999995 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576497e1-daeb-408a-a4ff-8a9d232e50dc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:15.039911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65183461-82b9-4924-be20-5a2fe2ab57db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:15.039911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65183461-82b9-4924-be20-5a2fe2ab57db admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:15.040478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65183461-82b9-4924-be20-5a2fe2ab57db admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:15.042235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65183461-82b9-4924-be20-5a2fe2ab57db admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:15.043892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65183461-82b9-4924-be20-5a2fe2ab57db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:15.047506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65183461-82b9-4924-be20-5a2fe2ab57db admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:15.050342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:15.051397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:15.051743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:15.052109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] PATH_INFO: `/v3/projects/5c339aeed9774c598bf07c6c10c0c551` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:15.055963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c339aeed9774c598bf07c6c10c0c551)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:15.058341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:15.105274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 860/1717] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:14 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:15.106427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65183461-82b9-4924-be20-5a2fe2ab57db admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:15.138199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65183461-82b9-4924-be20-5a2fe2ab57db admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:15.158405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4fae1-33de-4d09-976b-b334a6c607f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:15.162469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4fae1-33de-4d09-976b-b334a6c607f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:15.162780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4fae1-33de-4d09-976b-b334a6c607f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:15.165855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4fae1-33de-4d09-976b-b334a6c607f5 admin admin] PATH_INFO: `/v3/projects/f70c0761172f446cad9a73670a54eaec/users/0c86934c6a8249ec9f740cacc2970eb7/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:15.171411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4fae1-33de-4d09-976b-b334a6c607f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70c0761172f446cad9a73670a54eaec, user_id=0c86934c6a8249ec9f740cacc2970eb7, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:15.183616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4fae1-33de-4d09-976b-b334a6c607f5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:15.205751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c339aeed9774c598bf07c6c10c0c551', 'request_id': 'req-efe0877c-95f1-4602-96ca-ce15aa9ebc02'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.223427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 861/1718] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:15 2022] PUT /identity/v3/projects/f70c0761172f446cad9a73670a54eaec/users/0c86934c6a8249ec9f740cacc2970eb7/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:15.239835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4fae1-33de-4d09-976b-b334a6c607f5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:15.240677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c339aeed9774c598bf07c6c10c0c551', 'request_id': 'req-efe0877c-95f1-4602-96ca-ce15aa9ebc02'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.262560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c339aeed9774c598bf07c6c10c0c551', 'request_id': 'req-efe0877c-95f1-4602-96ca-ce15aa9ebc02'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.284851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:15.300222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] The token cache is being invalidate because project 5c339aeed9774c598bf07c6c10c0c551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:15.300588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.309738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-efe0877c-95f1-4602-96ca-ce15aa9ebc02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.322459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 858/1719] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:14 2022] DELETE /identity/v3/projects/5c339aeed9774c598bf07c6c10c0c551 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:15.325849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71b8e2-7e45-4c28-966b-d28f35750456 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:15.325849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71b8e2-7e45-4c28-966b-d28f35750456 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:15.326181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71b8e2-7e45-4c28-966b-d28f35750456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:15.496592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4fae1-33de-4d09-976b-b334a6c607f5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:15.507624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bec56b-c620-46b1-879e-d3cabc70586c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:15.508404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bec56b-c620-46b1-879e-d3cabc70586c admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:15.508727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bec56b-c620-46b1-879e-d3cabc70586c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:15.509040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bec56b-c620-46b1-879e-d3cabc70586c admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:15.510028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bec56b-c620-46b1-879e-d3cabc70586c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:15.514329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bec56b-c620-46b1-879e-d3cabc70586c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:15.534948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 862/1720] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:15.543819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9a71b8e2-7e45-4c28-966b-d28f35750456 None None] MFA Rules not processed for user `0c86934c6a8249ec9f740cacc2970eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:15.546449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bec56b-c620-46b1-879e-d3cabc70586c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:15.553550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9a71b8e2-7e45-4c28-966b-d28f35750456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:15.569353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-49bec56b-c620-46b1-879e-d3cabc70586c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:15.609035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bec56b-c620-46b1-879e-d3cabc70586c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:15.615275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:15.616601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:15.616887 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:15.617255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] PATH_INFO: `/v3/users/8a03fd0c0a614045848c7cd882e5ddc0` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:15.618837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a03fd0c0a614045848c7cd882e5ddc0)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:15.620838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:15.773213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a03fd0c0a614045848c7cd882e5ddc0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.812421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 859/1721] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:15.816500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71b8e2-7e45-4c28-966b-d28f35750456 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:15.818745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a03fd0c0a614045848c7cd882e5ddc0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.820874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9a71b8e2-7e45-4c28-966b-d28f35750456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:15.844033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a03fd0c0a614045848c7cd882e5ddc0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.869599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71b8e2-7e45-4c28-966b-d28f35750456 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:15.869831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a03fd0c0a614045848c7cd882e5ddc0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.870643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a03fd0c0a614045848c7cd882e5ddc0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:15.884298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44719376-817d-455f-86f8-1b505e1cf93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:15.885523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44719376-817d-455f-86f8-1b505e1cf93c admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:15.885891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44719376-817d-455f-86f8-1b505e1cf93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:15.886196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44719376-817d-455f-86f8-1b505e1cf93c admin admin] PATH_INFO: `/v3/projects/0403c19a701e49cca506c39a6f78118d/users/0739fcdb35b7403e9767abc5db1abebe/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:15.901882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 863/1722] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:15 2022] DELETE /identity/v3/users/8a03fd0c0a614045848c7cd882e5ddc0 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:15.903405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:15.908100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:15.926884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44719376-817d-455f-86f8-1b505e1cf93c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0403c19a701e49cca506c39a6f78118d, user_id=0739fcdb35b7403e9767abc5db1abebe, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:15.932722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44719376-817d-455f-86f8-1b505e1cf93c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:15.969388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 860/1723] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:15 2022] PUT /identity/v3/projects/0403c19a701e49cca506c39a6f78118d/users/0739fcdb35b7403e9767abc5db1abebe/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:15.975458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44719376-817d-455f-86f8-1b505e1cf93c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:15.985124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-44719376-817d-455f-86f8-1b505e1cf93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:16.007219 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:16.187660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ee134-569e-4a34-ad96-85afbb3b9f80 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:16.247015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44719376-817d-455f-86f8-1b505e1cf93c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:16.265988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe5e8c-7688-48de-aea4-7bc9684ac49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:16.271031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe5e8c-7688-48de-aea4-7bc9684ac49e admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:16.271312 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe5e8c-7688-48de-aea4-7bc9684ac49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:16.271716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe5e8c-7688-48de-aea4-7bc9684ac49e admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:16.272586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe5e8c-7688-48de-aea4-7bc9684ac49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:16.284353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe5e8c-7688-48de-aea4-7bc9684ac49e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:16.351005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 861/1724] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:16.361423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe5e8c-7688-48de-aea4-7bc9684ac49e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:16.383771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe5e8c-7688-48de-aea4-7bc9684ac49e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:16.391780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a332b-81ec-4a40-bd64-fa0a8320c71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:16.393653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a332b-81ec-4a40-bd64-fa0a8320c71c admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:16.394277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a332b-81ec-4a40-bd64-fa0a8320c71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:16.394686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a332b-81ec-4a40-bd64-fa0a8320c71c admin admin] PATH_INFO: `/v3/projects/0403c19a701e49cca506c39a6f78118d/users/0739fcdb35b7403e9767abc5db1abebe/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:16.401562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a332b-81ec-4a40-bd64-fa0a8320c71c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0403c19a701e49cca506c39a6f78118d, user_id=0739fcdb35b7403e9767abc5db1abebe, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:16.413673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a332b-81ec-4a40-bd64-fa0a8320c71c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:16.464596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 862/1725] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:16 2022] PUT /identity/v3/projects/0403c19a701e49cca506c39a6f78118d/users/0739fcdb35b7403e9767abc5db1abebe/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:16.466329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5b849-25b0-42e3-ba56-e2d4cd943027 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:16.466684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5b849-25b0-42e3-ba56-e2d4cd943027 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:16.466975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5b849-25b0-42e3-ba56-e2d4cd943027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:16.504878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:16.636944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1136dc-3882-412f-987c-12e6677819db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:16.639841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1136dc-3882-412f-987c-12e6677819db service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:16.644569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1136dc-3882-412f-987c-12e6677819db service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:16.645247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1136dc-3882-412f-987c-12e6677819db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:16.652384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8d1136dc-3882-412f-987c-12e6677819db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:16.898316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1136dc-3882-412f-987c-12e6677819db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:16.900234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1136dc-3882-412f-987c-12e6677819db service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:16.915916 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 864/1726] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 1014 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:17.135907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-10d5b849-25b0-42e3-ba56-e2d4cd943027 None None] MFA Rules not processed for user `0739fcdb35b7403e9767abc5db1abebe`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:17.137656 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-10d5b849-25b0-42e3-ba56-e2d4cd943027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:17.405650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82d6dd-2127-4967-8d12-6e9dbfc16824 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:17.406874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82d6dd-2127-4967-8d12-6e9dbfc16824 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:17.407504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82d6dd-2127-4967-8d12-6e9dbfc16824 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:17.413169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 865/1727] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 19:03:17 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:03:17.439802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 863/1728] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 970 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:17.481635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82d6dd-2127-4967-8d12-6e9dbfc16824 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:17.492907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2a82d6dd-2127-4967-8d12-6e9dbfc16824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:17.594391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb8a67-17bd-4ab2-9e66-0b023e388ed1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:17.594683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb8a67-17bd-4ab2-9e66-0b023e388ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:17.595062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb8a67-17bd-4ab2-9e66-0b023e388ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:17.638753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-12cb8a67-17bd-4ab2-9e66-0b023e388ed1 None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:17.640576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-12cb8a67-17bd-4ab2-9e66-0b023e388ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:17.878576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82d6dd-2127-4967-8d12-6e9dbfc16824 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:17.908513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccfb10-2dd0-4cf8-b5b4-cc37cd791c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:17.917067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccfb10-2dd0-4cf8-b5b4-cc37cd791c5e service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:17.920264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccfb10-2dd0-4cf8-b5b4-cc37cd791c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:17.923163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccfb10-2dd0-4cf8-b5b4-cc37cd791c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:17.930730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 864/1729] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 19:03:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:18.026588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccfb10-2dd0-4cf8-b5b4-cc37cd791c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:18.030186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccfb10-2dd0-4cf8-b5b4-cc37cd791c5e service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:18.080932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 866/1730] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:18.442742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb8a67-17bd-4ab2-9e66-0b023e388ed1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:18.465067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb8a67-17bd-4ab2-9e66-0b023e388ed1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:18.478938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a2518-f258-48ac-bfba-1ce4a213e61c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:18.480473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a2518-f258-48ac-bfba-1ce4a213e61c service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:18.485453 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a2518-f258-48ac-bfba-1ce4a213e61c service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:18.487310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a2518-f258-48ac-bfba-1ce4a213e61c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:18.494909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-374a2518-f258-48ac-bfba-1ce4a213e61c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:18.521325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:18.863115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a2518-f258-48ac-bfba-1ce4a213e61c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:18.869424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a2518-f258-48ac-bfba-1ce4a213e61c service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:18.915326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 865/1731] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:03:19.680557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccfb10-2dd0-4cf8-b5b4-cc37cd791c5e service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:19.684294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-05ccfb10-2dd0-4cf8-b5b4-cc37cd791c5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:19.935649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccfb10-2dd0-4cf8-b5b4-cc37cd791c5e service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:19.949071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:19.952367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:19.954187 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:19.954824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] PATH_INFO: `/v3/users/da5614a3a21942e6a67bfb830c66139a` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:19.959862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5614a3a21942e6a67bfb830c66139a)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:19.965459 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:20.056825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:20.119512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5614a3a21942e6a67bfb830c66139a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:20.201231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5614a3a21942e6a67bfb830c66139a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:20.222535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5614a3a21942e6a67bfb830c66139a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:20.257324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5614a3a21942e6a67bfb830c66139a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:20.262007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5614a3a21942e6a67bfb830c66139a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:20.285738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 867/1732] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:19 2022] DELETE /identity/v3/users/da5614a3a21942e6a67bfb830c66139a => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:20.291702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e998c74-0af0-45c3-b584-83b57cad6ef1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:20.293368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e998c74-0af0-45c3-b584-83b57cad6ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:20.293898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e998c74-0af0-45c3-b584-83b57cad6ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:20.336826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1e998c74-0af0-45c3-b584-83b57cad6ef1 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:20.356482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1e998c74-0af0-45c3-b584-83b57cad6ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:20.527876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 866/1733] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:20.710282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:20.722194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:20.902595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e998c74-0af0-45c3-b584-83b57cad6ef1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:20.904514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1e998c74-0af0-45c3-b584-83b57cad6ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:20.925574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7a1e7-f3a0-4ba8-a7b1-028b3e3e955d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:20.931994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e998c74-0af0-45c3-b584-83b57cad6ef1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:20.947338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed61892-90cb-4e0b-81a4-7f9de5059afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:20.948540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed61892-90cb-4e0b-81a4-7f9de5059afa service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:20.949556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed61892-90cb-4e0b-81a4-7f9de5059afa service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:20.954615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed61892-90cb-4e0b-81a4-7f9de5059afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:20.968382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:20.972751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:20.974334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:20.974759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] PATH_INFO: `/v3/projects/8f4cf23fdf7849fbb84e534df9ef08e6` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:20.979928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f4cf23fdf7849fbb84e534df9ef08e6)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:20.987063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:20.996143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed61892-90cb-4e0b-81a4-7f9de5059afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:20.999765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed61892-90cb-4e0b-81a4-7f9de5059afa service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:21.037072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 868/1734] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:21.132850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f4cf23fdf7849fbb84e534df9ef08e6', 'request_id': 'req-4fdd0df2-d864-415e-8612-344704a500c9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:21.143488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f4cf23fdf7849fbb84e534df9ef08e6', 'request_id': 'req-4fdd0df2-d864-415e-8612-344704a500c9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:21.156649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f4cf23fdf7849fbb84e534df9ef08e6', 'request_id': 'req-4fdd0df2-d864-415e-8612-344704a500c9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:21.171629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] The token cache is being invalidate because project 8f4cf23fdf7849fbb84e534df9ef08e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:21.171977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:21.172701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4fdd0df2-d864-415e-8612-344704a500c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:21.175814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 867/1735] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:20 2022] DELETE /identity/v3/projects/8f4cf23fdf7849fbb84e534df9ef08e6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:21.183997 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db878d71-b087-4fab-afed-44e6b5343f44 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:21.184278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db878d71-b087-4fab-afed-44e6b5343f44 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:21.184717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db878d71-b087-4fab-afed-44e6b5343f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:21.223319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-db878d71-b087-4fab-afed-44e6b5343f44 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:21.226396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-db878d71-b087-4fab-afed-44e6b5343f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:21.314473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 869/1736] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:21.319625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2bc0-0f2c-4276-b34e-6898a6e87a3d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:21.320454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2bc0-0f2c-4276-b34e-6898a6e87a3d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:21.323396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2bc0-0f2c-4276-b34e-6898a6e87a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:21.352221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-584a2bc0-0f2c-4276-b34e-6898a6e87a3d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:21.353513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-584a2bc0-0f2c-4276-b34e-6898a6e87a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:21.365473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 868/1737] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:21.369191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db878d71-b087-4fab-afed-44e6b5343f44 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:21.383813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db878d71-b087-4fab-afed-44e6b5343f44 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:21.390292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe277d-2403-47e0-affc-a5fe27a7c251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:21.391144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe277d-2403-47e0-affc-a5fe27a7c251 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:21.391456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe277d-2403-47e0-affc-a5fe27a7c251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:21.391815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe277d-2403-47e0-affc-a5fe27a7c251 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:21.392811 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe277d-2403-47e0-affc-a5fe27a7c251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:21.393214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe277d-2403-47e0-affc-a5fe27a7c251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:21.395272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe277d-2403-47e0-affc-a5fe27a7c251 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:21.413157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 870/1738] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:21.417024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2bc0-0f2c-4276-b34e-6898a6e87a3d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:21.431637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2bc0-0f2c-4276-b34e-6898a6e87a3d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:21.437209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d528f905-9b90-4be6-b1fc-a3a7e97aede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:21.438550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528f905-9b90-4be6-b1fc-a3a7e97aede0 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:21.439046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528f905-9b90-4be6-b1fc-a3a7e97aede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:21.439318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528f905-9b90-4be6-b1fc-a3a7e97aede0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:21.440673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d528f905-9b90-4be6-b1fc-a3a7e97aede0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:21.443269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d528f905-9b90-4be6-b1fc-a3a7e97aede0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:21.469950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:21.479008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 869/1739] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:21 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:21.484957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe277d-2403-47e0-affc-a5fe27a7c251 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:21.496327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe277d-2403-47e0-affc-a5fe27a7c251 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:21.551235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:21.578222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ae7d1-446f-45aa-b0df-7cd56b31399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:21.578911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ae7d1-446f-45aa-b0df-7cd56b31399c admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:21.579199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ae7d1-446f-45aa-b0df-7cd56b31399c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:21.579482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ae7d1-446f-45aa-b0df-7cd56b31399c admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:21.580269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ae7d1-446f-45aa-b0df-7cd56b31399c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:21.582787 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ae7d1-446f-45aa-b0df-7cd56b31399c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:21.605955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 871/1740] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:21 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:21.611975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d528f905-9b90-4be6-b1fc-a3a7e97aede0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:21.631562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d528f905-9b90-4be6-b1fc-a3a7e97aede0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:21.638389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766a32c-0c2b-489f-90ad-d11743ce9757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:21.639704 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766a32c-0c2b-489f-90ad-d11743ce9757 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:21.640317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766a32c-0c2b-489f-90ad-d11743ce9757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:21.640659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766a32c-0c2b-489f-90ad-d11743ce9757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:21.641648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766a32c-0c2b-489f-90ad-d11743ce9757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:21.644590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766a32c-0c2b-489f-90ad-d11743ce9757 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:21.662411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 870/1741] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:21.667522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ae7d1-446f-45aa-b0df-7cd56b31399c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:21.686299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ae7d1-446f-45aa-b0df-7cd56b31399c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:21.697122 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b929506-ce8d-48fe-b88f-cc642b00cb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:21.698622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b929506-ce8d-48fe-b88f-cc642b00cb20 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:21.698737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b929506-ce8d-48fe-b88f-cc642b00cb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:21.699273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b929506-ce8d-48fe-b88f-cc642b00cb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:21.700270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b929506-ce8d-48fe-b88f-cc642b00cb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:21.703280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b929506-ce8d-48fe-b88f-cc642b00cb20 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:21.723838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 872/1742] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:21.728933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766a32c-0c2b-489f-90ad-d11743ce9757 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:21.759945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766a32c-0c2b-489f-90ad-d11743ce9757 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:21.767675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9aa0c-2373-4097-b606-0af69bb79ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:21.768865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9aa0c-2373-4097-b606-0af69bb79ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:21.769332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9aa0c-2373-4097-b606-0af69bb79ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:21.769741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9aa0c-2373-4097-b606-0af69bb79ba4 admin admin] PATH_INFO: `/v3/projects/f5798fe4719f4d729b32aad87b5483e3/users/1093077a5e0e473888c5c9cff33ef6b0/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:21.790552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9aa0c-2373-4097-b606-0af69bb79ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5798fe4719f4d729b32aad87b5483e3, user_id=1093077a5e0e473888c5c9cff33ef6b0, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:21.793297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9aa0c-2373-4097-b606-0af69bb79ba4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:21.820329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 871/1743] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:21 2022] PUT /identity/v3/projects/f5798fe4719f4d729b32aad87b5483e3/users/1093077a5e0e473888c5c9cff33ef6b0/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:21.828935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b929506-ce8d-48fe-b88f-cc642b00cb20 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:21.836700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0b929506-ce8d-48fe-b88f-cc642b00cb20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:21.845304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ab0b5-e99b-481e-9dc7-1af6e5667e44 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:21.845807 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ab0b5-e99b-481e-9dc7-1af6e5667e44 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:21.846298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ab0b5-e99b-481e-9dc7-1af6e5667e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:21.891080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6b0ab0b5-e99b-481e-9dc7-1af6e5667e44 None None] MFA Rules not processed for user `1093077a5e0e473888c5c9cff33ef6b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:21.892896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6b0ab0b5-e99b-481e-9dc7-1af6e5667e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:21.971846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b929506-ce8d-48fe-b88f-cc642b00cb20 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:22.015571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 872/1744] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:22.022047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ab0b5-e99b-481e-9dc7-1af6e5667e44 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:22.181697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91099c27-b89f-478e-8121-d71216e165da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:22.183373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91099c27-b89f-478e-8121-d71216e165da service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:22.184034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91099c27-b89f-478e-8121-d71216e165da service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:22.184331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91099c27-b89f-478e-8121-d71216e165da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:22.187912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ab0b5-e99b-481e-9dc7-1af6e5667e44 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:22.201823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-91099c27-b89f-478e-8121-d71216e165da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:22.202434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893e509-9d32-4890-8280-9ddc53125b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:22.203950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893e509-9d32-4890-8280-9ddc53125b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:22.204522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893e509-9d32-4890-8280-9ddc53125b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:22.205949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893e509-9d32-4890-8280-9ddc53125b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:22.207369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893e509-9d32-4890-8280-9ddc53125b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:22.210522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893e509-9d32-4890-8280-9ddc53125b87 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:22.218009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91099c27-b89f-478e-8121-d71216e165da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:22.222932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91099c27-b89f-478e-8121-d71216e165da service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:22.249266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 873/1745] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:22.285264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 873/1746] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:22 2022] POST /identity/v3/projects => generated 387 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:22.290131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639c1b6-6f35-4b1f-942a-5fc58ac82027 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:22.290718 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639c1b6-6f35-4b1f-942a-5fc58ac82027 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:22.291188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639c1b6-6f35-4b1f-942a-5fc58ac82027 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:22.293564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 874/1747] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 19:03:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:03:22.298551 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893e509-9d32-4890-8280-9ddc53125b87 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:22.359063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c299083-c57a-41bd-9ff6-527844ee19d8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:22.359682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c299083-c57a-41bd-9ff6-527844ee19d8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:22.360110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c299083-c57a-41bd-9ff6-527844ee19d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:22.413357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9c299083-c57a-41bd-9ff6-527844ee19d8 None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:22.416779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9c299083-c57a-41bd-9ff6-527844ee19d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:22.453365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:22.551668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893e509-9d32-4890-8280-9ddc53125b87 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:22.573303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd3958-c210-43ee-8a06-8b8360659884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:22.574021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd3958-c210-43ee-8a06-8b8360659884 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:22.574317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd3958-c210-43ee-8a06-8b8360659884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:22.574606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd3958-c210-43ee-8a06-8b8360659884 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:22.575336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd3958-c210-43ee-8a06-8b8360659884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:22.577381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd3958-c210-43ee-8a06-8b8360659884 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:22.622714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 874/1748] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:22 2022] POST /identity/v3/users => generated 300 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:22.626593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd3958-c210-43ee-8a06-8b8360659884 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:22.641928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:22.657851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd3958-c210-43ee-8a06-8b8360659884 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:22.676568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60040719-0a92-47b9-8e0b-6b60f87538c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:22.677294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60040719-0a92-47b9-8e0b-6b60f87538c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:22.677587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60040719-0a92-47b9-8e0b-6b60f87538c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:22.677872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60040719-0a92-47b9-8e0b-6b60f87538c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:22.678627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60040719-0a92-47b9-8e0b-6b60f87538c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:22.680280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60040719-0a92-47b9-8e0b-6b60f87538c0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:22.717925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 875/1749] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:22.718904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60040719-0a92-47b9-8e0b-6b60f87538c0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:22.746507 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60040719-0a92-47b9-8e0b-6b60f87538c0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:22.749180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 875/1750] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 19:03:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:22.765775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb3391-8082-4d43-8373-c0c6c63f168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:22.769109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb3391-8082-4d43-8373-c0c6c63f168b admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:22.769421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb3391-8082-4d43-8373-c0c6c63f168b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:22.769711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb3391-8082-4d43-8373-c0c6c63f168b admin admin] PATH_INFO: `/v3/projects/77cfe7e01013404ca8b0ac9432ed28e6/users/1e329bed249c4fb7a4f6940e0ab2ae8f/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:22.782574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c299083-c57a-41bd-9ff6-527844ee19d8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:22.785782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9c299083-c57a-41bd-9ff6-527844ee19d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:22.796132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb3391-8082-4d43-8373-c0c6c63f168b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77cfe7e01013404ca8b0ac9432ed28e6, user_id=1e329bed249c4fb7a4f6940e0ab2ae8f, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:22.800884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb3391-8082-4d43-8373-c0c6c63f168b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:22.824219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c299083-c57a-41bd-9ff6-527844ee19d8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:22.836573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 876/1751] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:22 2022] PUT /identity/v3/projects/77cfe7e01013404ca8b0ac9432ed28e6/users/1e329bed249c4fb7a4f6940e0ab2ae8f/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:22.847050 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb3391-8082-4d43-8373-c0c6c63f168b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:22.853288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-beeb3391-8082-4d43-8373-c0c6c63f168b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:22.990888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6474a-a165-4b8e-9495-3171b7e9312b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:22.991362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6474a-a165-4b8e-9495-3171b7e9312b service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:22.991805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6474a-a165-4b8e-9495-3171b7e9312b service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:22.992084 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6474a-a165-4b8e-9495-3171b7e9312b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:23.014154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6474a-a165-4b8e-9495-3171b7e9312b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:23.016051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6474a-a165-4b8e-9495-3171b7e9312b service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:23.049282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 876/1752] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:03:23.050320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6474a-a165-4b8e-9495-3171b7e9312b service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:23.054586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb3391-8082-4d43-8373-c0c6c63f168b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:23.072331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:23.072660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:23.073119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:23.073404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] PATH_INFO: `/v3/users/e222c1107db143cfb25c3bf297e90e70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:23.074069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6474a-a165-4b8e-9495-3171b7e9312b service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:23.078432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e222c1107db143cfb25c3bf297e90e70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:23.081962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b527e-51c0-4ace-af63-5296f5cfb571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:23.082336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:23.083072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b527e-51c0-4ace-af63-5296f5cfb571 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:23.083441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b527e-51c0-4ace-af63-5296f5cfb571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:23.083881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b527e-51c0-4ace-af63-5296f5cfb571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:23.085200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b527e-51c0-4ace-af63-5296f5cfb571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:23.087516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b527e-51c0-4ace-af63-5296f5cfb571 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:23.113500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 877/1753] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:23.144113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b527e-51c0-4ace-af63-5296f5cfb571 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:23.147643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-507b527e-51c0-4ace-af63-5296f5cfb571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:23.183906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b527e-51c0-4ace-af63-5296f5cfb571 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:23.200741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:23.202028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:23.202028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:23.202290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] PATH_INFO: `/v3/projects/6995d7b9eeab49bf9888ebe4039ad54c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:23.203567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6995d7b9eeab49bf9888ebe4039ad54c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:23.205623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:23.211388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e222c1107db143cfb25c3bf297e90e70'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.245502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e222c1107db143cfb25c3bf297e90e70'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.268471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e222c1107db143cfb25c3bf297e90e70'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.271406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e222c1107db143cfb25c3bf297e90e70'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.302427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e222c1107db143cfb25c3bf297e90e70'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.319348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 877/1754] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:22 2022] DELETE /identity/v3/users/e222c1107db143cfb25c3bf297e90e70 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:23.319348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:23.319348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:23.337670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6995d7b9eeab49bf9888ebe4039ad54c', 'request_id': 'req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.347362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:23.357460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6995d7b9eeab49bf9888ebe4039ad54c', 'request_id': 'req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.368747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6995d7b9eeab49bf9888ebe4039ad54c', 'request_id': 'req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.402065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] The token cache is being invalidate because project 6995d7b9eeab49bf9888ebe4039ad54c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:23.404001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.406372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-beed8d05-59a1-4ea1-8c6e-a7bfc60361b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.421226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 878/1755] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:23 2022] DELETE /identity/v3/projects/6995d7b9eeab49bf9888ebe4039ad54c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:23.422831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c24f0d-d616-4af2-9c9a-229d74e739e2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:23.423131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c24f0d-d616-4af2-9c9a-229d74e739e2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:23.423431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c24f0d-d616-4af2-9c9a-229d74e739e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:23.441709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e5df-4121-4654-9c20-4c54a1a866ea admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:23.448176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284a48f-a9e5-4897-a911-0450c94709b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:23.449150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284a48f-a9e5-4897-a911-0450c94709b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:23.449438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284a48f-a9e5-4897-a911-0450c94709b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:23.449720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284a48f-a9e5-4897-a911-0450c94709b8 admin admin] PATH_INFO: `/v3/projects/77cfe7e01013404ca8b0ac9432ed28e6/users/1e329bed249c4fb7a4f6940e0ab2ae8f/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:23.452143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284a48f-a9e5-4897-a911-0450c94709b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77cfe7e01013404ca8b0ac9432ed28e6, user_id=1e329bed249c4fb7a4f6940e0ab2ae8f, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:23.454949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284a48f-a9e5-4897-a911-0450c94709b8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:23.457240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b5c24f0d-d616-4af2-9c9a-229d74e739e2 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:23.459499 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b5c24f0d-d616-4af2-9c9a-229d74e739e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:23.471766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 878/1756] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:23 2022] PUT /identity/v3/projects/77cfe7e01013404ca8b0ac9432ed28e6/users/1e329bed249c4fb7a4f6940e0ab2ae8f/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:23.473165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284a48f-a9e5-4897-a911-0450c94709b8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:23.484754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4284a48f-a9e5-4897-a911-0450c94709b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:23.504545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:23.564387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 879/1757] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:23.570780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549fbf8-b7a7-4289-8548-3ddbff77afc2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:23.571450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549fbf8-b7a7-4289-8548-3ddbff77afc2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:23.571914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549fbf8-b7a7-4289-8548-3ddbff77afc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:23.589786 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284a48f-a9e5-4897-a911-0450c94709b8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:23.595522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:23.596747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:23.597030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:23.597344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] PATH_INFO: `/v3/users/0c86934c6a8249ec9f740cacc2970eb7` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:23.598746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c86934c6a8249ec9f740cacc2970eb7)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:23.600731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:23.616670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5549fbf8-b7a7-4289-8548-3ddbff77afc2 None None] MFA Rules not processed for user `1e329bed249c4fb7a4f6940e0ab2ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:23.618825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5549fbf8-b7a7-4289-8548-3ddbff77afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:23.689283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c86934c6a8249ec9f740cacc2970eb7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.712849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c86934c6a8249ec9f740cacc2970eb7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.728609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c86934c6a8249ec9f740cacc2970eb7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.729749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c86934c6a8249ec9f740cacc2970eb7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.742421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c86934c6a8249ec9f740cacc2970eb7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:23.749969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 879/1758] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:23 2022] DELETE /identity/v3/users/0c86934c6a8249ec9f740cacc2970eb7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:23.754509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:23.756288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:23.770552 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 880/1759] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:23.794105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549fbf8-b7a7-4289-8548-3ddbff77afc2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:23.797864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5549fbf8-b7a7-4289-8548-3ddbff77afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:23.834614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:23.898328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0e600-e9cd-424c-8528-f5e618a5c10e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:23.909842 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:23.910560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:23.910926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:23.911199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:23.916243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:24.032083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549fbf8-b7a7-4289-8548-3ddbff77afc2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:24.119553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:24.120353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:24.120883 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:24.121288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:24.130450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:24.165964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:24.169596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:24.171948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:24.202601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 880/1760] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:24.258692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:24.260364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:24.286624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 881/1761] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:24.465644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:24.470325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:24.490496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f19c3-2a79-4340-9a1e-fbb89ba8b932 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:24.502272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:24.504012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:24.504424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:24.504813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] PATH_INFO: `/v3/users/1093077a5e0e473888c5c9cff33ef6b0` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:24.506741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1093077a5e0e473888c5c9cff33ef6b0)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:24.509372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:24.596708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:24.599387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:24.621553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22b60a-6996-4730-b631-f069c282165f service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:24.627296 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:24.628188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:24.628475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:24.628756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] PATH_INFO: `/v3/projects/f70c0761172f446cad9a73670a54eaec` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:24.630172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70c0761172f446cad9a73670a54eaec)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:24.630494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1093077a5e0e473888c5c9cff33ef6b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.632391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:24.733008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70c0761172f446cad9a73670a54eaec', 'request_id': 'req-a3226327-937a-4bce-b9a0-2e42fedce886'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.747059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70c0761172f446cad9a73670a54eaec', 'request_id': 'req-a3226327-937a-4bce-b9a0-2e42fedce886'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.750977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1093077a5e0e473888c5c9cff33ef6b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.769227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70c0761172f446cad9a73670a54eaec', 'request_id': 'req-a3226327-937a-4bce-b9a0-2e42fedce886'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.772546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1093077a5e0e473888c5c9cff33ef6b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.775930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1093077a5e0e473888c5c9cff33ef6b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.799599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3ee9f46-4029-4d22-9471-8e6843822635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1093077a5e0e473888c5c9cff33ef6b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.802403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] The token cache is being invalidate because project f70c0761172f446cad9a73670a54eaec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:24.803291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.805033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:24.811385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 882/1762] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:24 2022] DELETE /identity/v3/projects/f70c0761172f446cad9a73670a54eaec => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:24.811385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:24.813698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:24.820711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 881/1763] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:24 2022] DELETE /identity/v3/users/1093077a5e0e473888c5c9cff33ef6b0 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:24.828970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e6f92-803f-40b3-a5fb-a445e90de69c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:24.828970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e6f92-803f-40b3-a5fb-a445e90de69c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:24.828970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e6f92-803f-40b3-a5fb-a445e90de69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:24.889422 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-375e6f92-803f-40b3-a5fb-a445e90de69c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:24.892308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-375e6f92-803f-40b3-a5fb-a445e90de69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:25.020196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3226327-937a-4bce-b9a0-2e42fedce886 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:25.031173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:25.070614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 882/1764] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:25.072613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c34888-6ae9-4729-8907-5a938463bd8a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.077982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c34888-6ae9-4729-8907-5a938463bd8a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.078404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c34888-6ae9-4729-8907-5a938463bd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.102420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e1c34888-6ae9-4729-8907-5a938463bd8a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:25.103812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e1c34888-6ae9-4729-8907-5a938463bd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:25.113442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 883/1765] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:25.114894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914363a8-f5b9-48d2-9fdc-20db8f146a4b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.115261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914363a8-f5b9-48d2-9fdc-20db8f146a4b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.115620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914363a8-f5b9-48d2-9fdc-20db8f146a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.130084 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:25.162457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-914363a8-f5b9-48d2-9fdc-20db8f146a4b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:25.162842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:25.164054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.164648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.165008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-914363a8-f5b9-48d2-9fdc-20db8f146a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:25.165620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] PATH_INFO: `/v3/projects/5e839f79cc74463b8fff3e45c302c5e9` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.172783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e839f79cc74463b8fff3e45c302c5e9)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:25.177967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:25.179515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 884/1766] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:25.180465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914363a8-f5b9-48d2-9fdc-20db8f146a4b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:25.182833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-914363a8-f5b9-48d2-9fdc-20db8f146a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:25.257366 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e839f79cc74463b8fff3e45c302c5e9', 'request_id': 'req-a43fd8e1-fe61-4628-a015-14069db6c56b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.264354 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914363a8-f5b9-48d2-9fdc-20db8f146a4b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:25.266035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e839f79cc74463b8fff3e45c302c5e9', 'request_id': 'req-a43fd8e1-fe61-4628-a015-14069db6c56b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.286883 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e839f79cc74463b8fff3e45c302c5e9', 'request_id': 'req-a43fd8e1-fe61-4628-a015-14069db6c56b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.310955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] The token cache is being invalidate because project 5e839f79cc74463b8fff3e45c302c5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:25.311502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.313546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.317051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 883/1767] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:24 2022] DELETE /identity/v3/projects/5e839f79cc74463b8fff3e45c302c5e9 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:25.324953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:25.327218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:25.356001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e75039-d540-4fec-a78e-fd3037babab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:25.356681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e75039-d540-4fec-a78e-fd3037babab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.356969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e75039-d540-4fec-a78e-fd3037babab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.357286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e75039-d540-4fec-a78e-fd3037babab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.359506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-30e75039-d540-4fec-a78e-fd3037babab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:25.431630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fd8e1-fe61-4628-a015-14069db6c56b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:25.436493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:25.437274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.437574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.437847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.438580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:25.438908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:25.441407 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:25.454496 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 884/1768] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:25.455416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:25.460115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:25.472362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e75039-d540-4fec-a78e-fd3037babab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:25.474000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e75039-d540-4fec-a78e-fd3037babab8 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:25.475907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b3f1a-e4db-4da9-8a13-4d13d0c3c52c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:25.483874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:25.484764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.485135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.485423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] PATH_INFO: `/v3/users/0739fcdb35b7403e9767abc5db1abebe` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.486964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 885/1769] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:25.487436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0739fcdb35b7403e9767abc5db1abebe)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:25.487691 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e75039-d540-4fec-a78e-fd3037babab8 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:25.489470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:25.499097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e75039-d540-4fec-a78e-fd3037babab8 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:25.504606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e7823-11a9-4e7d-b97a-e2e864625be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:25.505582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e7823-11a9-4e7d-b97a-e2e864625be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.505879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e7823-11a9-4e7d-b97a-e2e864625be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.506159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e7823-11a9-4e7d-b97a-e2e864625be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.506874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e7823-11a9-4e7d-b97a-e2e864625be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:25.509405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e7823-11a9-4e7d-b97a-e2e864625be4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:25.542536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 886/1770] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:25 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:25.546526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e7823-11a9-4e7d-b97a-e2e864625be4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:25.574056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:25.579386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0739fcdb35b7403e9767abc5db1abebe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.597481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0739fcdb35b7403e9767abc5db1abebe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.606685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0739fcdb35b7403e9767abc5db1abebe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.608387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:25.620240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0739fcdb35b7403e9767abc5db1abebe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.623144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0739fcdb35b7403e9767abc5db1abebe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:25.636328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 885/1771] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:25 2022] DELETE /identity/v3/users/0739fcdb35b7403e9767abc5db1abebe => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:25.647157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e7823-11a9-4e7d-b97a-e2e864625be4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:25.740476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465ca89-27dc-4e3c-93f7-5330d04f4711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:25.741399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465ca89-27dc-4e3c-93f7-5330d04f4711 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.741692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465ca89-27dc-4e3c-93f7-5330d04f4711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.741981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465ca89-27dc-4e3c-93f7-5330d04f4711 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.743493 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465ca89-27dc-4e3c-93f7-5330d04f4711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:25.746600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465ca89-27dc-4e3c-93f7-5330d04f4711 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:25.777142 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 887/1772] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:25 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:25.778884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:25.780423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:25.796102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f820569-9c87-4e3f-b05a-e58d129ea5aa admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:25.804750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd4296-fb48-49c3-be10-4a96415f0685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:25.806572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd4296-fb48-49c3-be10-4a96415f0685 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.806717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd4296-fb48-49c3-be10-4a96415f0685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.807440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd4296-fb48-49c3-be10-4a96415f0685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.808971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd4296-fb48-49c3-be10-4a96415f0685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:25.812446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd4296-fb48-49c3-be10-4a96415f0685 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:25.833874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 886/1773] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:25.837985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465ca89-27dc-4e3c-93f7-5330d04f4711 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:25.854410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465ca89-27dc-4e3c-93f7-5330d04f4711 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:25.862959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f36120-6bd1-459c-ae35-c9d486035c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:25.864150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f36120-6bd1-459c-ae35-c9d486035c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.864668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f36120-6bd1-459c-ae35-c9d486035c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.865215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f36120-6bd1-459c-ae35-c9d486035c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.866611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f36120-6bd1-459c-ae35-c9d486035c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:25.869800 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f36120-6bd1-459c-ae35-c9d486035c8a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:25.886910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 888/1774] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:25.890495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd4296-fb48-49c3-be10-4a96415f0685 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:25.901364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd4296-fb48-49c3-be10-4a96415f0685 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:25.906257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03b6bc-77db-499f-8f78-7f8b4d80f4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:25.907172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03b6bc-77db-499f-8f78-7f8b4d80f4f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.907463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03b6bc-77db-499f-8f78-7f8b4d80f4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.907761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03b6bc-77db-499f-8f78-7f8b4d80f4f2 admin admin] PATH_INFO: `/v3/projects/aa045b7f376740d7a8e6e4308075e4e7/users/81c6b704fb5b4719a89a7d752b211962/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:25.927309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03b6bc-77db-499f-8f78-7f8b4d80f4f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa045b7f376740d7a8e6e4308075e4e7, user_id=81c6b704fb5b4719a89a7d752b211962, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:25.931384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03b6bc-77db-499f-8f78-7f8b4d80f4f2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:25.962478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 887/1775] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:25 2022] PUT /identity/v3/projects/aa045b7f376740d7a8e6e4308075e4e7/users/81c6b704fb5b4719a89a7d752b211962/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:25.966886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22143d-2cd6-4729-8559-46d7076573cc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:25.967378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22143d-2cd6-4729-8559-46d7076573cc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:25.967734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22143d-2cd6-4729-8559-46d7076573cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.034461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5c22143d-2cd6-4729-8559-46d7076573cc None None] MFA Rules not processed for user `81c6b704fb5b4719a89a7d752b211962`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:26.035919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5c22143d-2cd6-4729-8559-46d7076573cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.121752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03b6bc-77db-499f-8f78-7f8b4d80f4f2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:26.124380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0d03b6bc-77db-499f-8f78-7f8b4d80f4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.139051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 889/1776] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:26.144590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22143d-2cd6-4729-8559-46d7076573cc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:26.213428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:26.249339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22143d-2cd6-4729-8559-46d7076573cc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:26.254679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b41e4-c5b4-4932-b7b8-397c066f2b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:26.255369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b41e4-c5b4-4932-b7b8-397c066f2b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:26.255657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b41e4-c5b4-4932-b7b8-397c066f2b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:26.255940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b41e4-c5b4-4932-b7b8-397c066f2b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.256795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b41e4-c5b4-4932-b7b8-397c066f2b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:26.259221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b41e4-c5b4-4932-b7b8-397c066f2b87 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:26.262600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03b6bc-77db-499f-8f78-7f8b4d80f4f2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:26.271029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:26.272002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:26.272910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:26.273275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] PATH_INFO: `/v3/projects/f5798fe4719f4d729b32aad87b5483e3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.275381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5798fe4719f4d729b32aad87b5483e3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:26.277684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:26.298306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 890/1777] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:26 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:26.300010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b41e4-c5b4-4932-b7b8-397c066f2b87 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:26.302587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a67b41e4-c5b4-4932-b7b8-397c066f2b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.365070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5798fe4719f4d729b32aad87b5483e3', 'request_id': 'req-f584ce2a-c72c-4c74-a577-c7becd260667'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.374348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5798fe4719f4d729b32aad87b5483e3', 'request_id': 'req-f584ce2a-c72c-4c74-a577-c7becd260667'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.386685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5798fe4719f4d729b32aad87b5483e3', 'request_id': 'req-f584ce2a-c72c-4c74-a577-c7becd260667'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.402517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] The token cache is being invalidate because project f5798fe4719f4d729b32aad87b5483e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:26.402827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.403504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.405227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 888/1778] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:26 2022] DELETE /identity/v3/projects/f5798fe4719f4d729b32aad87b5483e3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:26.406018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:26.413684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.417509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b41e4-c5b4-4932-b7b8-397c066f2b87 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:26.493518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:26.494682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:26.495040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:26.495581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] PATH_INFO: `/v3/projects/0403c19a701e49cca506c39a6f78118d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.497160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0403c19a701e49cca506c39a6f78118d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:26.499308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:26.509220 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:26.548507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584ce2a-c72c-4c74-a577-c7becd260667 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:26.572068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0403c19a701e49cca506c39a6f78118d', 'request_id': 'req-aeab2d91-2775-4645-976d-ebbb9382375c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.583690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0403c19a701e49cca506c39a6f78118d', 'request_id': 'req-aeab2d91-2775-4645-976d-ebbb9382375c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.601559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0403c19a701e49cca506c39a6f78118d', 'request_id': 'req-aeab2d91-2775-4645-976d-ebbb9382375c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.615187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:26.623537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] The token cache is being invalidate because project 0403c19a701e49cca506c39a6f78118d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:26.624525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.625908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.632127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 891/1779] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:26 2022] DELETE /identity/v3/projects/0403c19a701e49cca506c39a6f78118d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:26.633838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:26.636753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.667763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e516205-c204-48de-99f8-c361307551b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:26.669008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e516205-c204-48de-99f8-c361307551b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:26.669532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e516205-c204-48de-99f8-c361307551b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:26.670035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e516205-c204-48de-99f8-c361307551b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.671414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e516205-c204-48de-99f8-c361307551b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:26.675441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e516205-c204-48de-99f8-c361307551b7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:26.707701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 889/1780] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:26 2022] POST /identity/v3/users => generated 296 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:26.711064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250f5e6-a1d9-4f1d-b79c-c5569bdbee33 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:26.711645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250f5e6-a1d9-4f1d-b79c-c5569bdbee33 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:26.712089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250f5e6-a1d9-4f1d-b79c-c5569bdbee33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.738656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0250f5e6-a1d9-4f1d-b79c-c5569bdbee33 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:26.741637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0250f5e6-a1d9-4f1d-b79c-c5569bdbee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.750596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab2d91-2775-4645-976d-ebbb9382375c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:26.754790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 890/1781] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:26.755893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250f5e6-a1d9-4f1d-b79c-c5569bdbee33 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:26.757713 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:26.757927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0250f5e6-a1d9-4f1d-b79c-c5569bdbee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.758658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:26.758942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:26.759224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] PATH_INFO: `/v3/users/1e329bed249c4fb7a4f6940e0ab2ae8f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.760790 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e329bed249c4fb7a4f6940e0ab2ae8f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:26.763454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:26.780630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250f5e6-a1d9-4f1d-b79c-c5569bdbee33 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:26.788814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8ba82-40cc-4680-b8c5-31d28cd26bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:26.789980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8ba82-40cc-4680-b8c5-31d28cd26bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:26.790479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8ba82-40cc-4680-b8c5-31d28cd26bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:26.790954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8ba82-40cc-4680-b8c5-31d28cd26bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.791807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8ba82-40cc-4680-b8c5-31d28cd26bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:26.794148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8ba82-40cc-4680-b8c5-31d28cd26bcc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:26.813165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 891/1782] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:26.813165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ababe8-54a1-4d9f-a95d-b4dd6a6a4191 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:26.814893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ababe8-54a1-4d9f-a95d-b4dd6a6a4191 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:26.814893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ababe8-54a1-4d9f-a95d-b4dd6a6a4191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.844171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e329bed249c4fb7a4f6940e0ab2ae8f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.851839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f0ababe8-54a1-4d9f-a95d-b4dd6a6a4191 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:26.855153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f0ababe8-54a1-4d9f-a95d-b4dd6a6a4191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.861055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e329bed249c4fb7a4f6940e0ab2ae8f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.868897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 892/1783] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:26.868897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ababe8-54a1-4d9f-a95d-b4dd6a6a4191 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:26.871076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e329bed249c4fb7a4f6940e0ab2ae8f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.871815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e329bed249c4fb7a4f6940e0ab2ae8f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.880367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ababe8-54a1-4d9f-a95d-b4dd6a6a4191 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:26.882459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e329bed249c4fb7a4f6940e0ab2ae8f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:26.884886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe0e9e-bfe7-49c6-9971-77396d8a8561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:26.885861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe0e9e-bfe7-49c6-9971-77396d8a8561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:26.886148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe0e9e-bfe7-49c6-9971-77396d8a8561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:26.886430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe0e9e-bfe7-49c6-9971-77396d8a8561 admin admin] PATH_INFO: `/v3/projects/e95169044859411e984841352bbf6dc6/users/2ded18b2e6764fb8b77fc4b625039324/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:26.890520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 892/1784] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:26 2022] DELETE /identity/v3/users/1e329bed249c4fb7a4f6940e0ab2ae8f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:26.891783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:26.894746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.905319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe0e9e-bfe7-49c6-9971-77396d8a8561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95169044859411e984841352bbf6dc6, user_id=2ded18b2e6764fb8b77fc4b625039324, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:26.908177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe0e9e-bfe7-49c6-9971-77396d8a8561 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:26.929844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 893/1785] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:26 2022] PUT /identity/v3/projects/e95169044859411e984841352bbf6dc6/users/2ded18b2e6764fb8b77fc4b625039324/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:26.935916 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe0e9e-bfe7-49c6-9971-77396d8a8561 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:26.941398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7bfe0e9e-bfe7-49c6-9971-77396d8a8561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:26.987880 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbec4fd-42af-422a-a59a-3b88d800258b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.034963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:27.088121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe0e9e-bfe7-49c6-9971-77396d8a8561 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.097633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482de037-f818-4d9e-a8d1-1745e3df08d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.098891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482de037-f818-4d9e-a8d1-1745e3df08d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.099487 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482de037-f818-4d9e-a8d1-1745e3df08d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.100070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482de037-f818-4d9e-a8d1-1745e3df08d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.102294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482de037-f818-4d9e-a8d1-1745e3df08d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.105660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482de037-f818-4d9e-a8d1-1745e3df08d9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.125317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec219cc-c451-4bbc-8984-863b03abbca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.126840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec219cc-c451-4bbc-8984-863b03abbca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.128008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec219cc-c451-4bbc-8984-863b03abbca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.128464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec219cc-c451-4bbc-8984-863b03abbca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.130489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec219cc-c451-4bbc-8984-863b03abbca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:27.130972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec219cc-c451-4bbc-8984-863b03abbca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.135107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 894/1786] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:03:27.135395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec219cc-c451-4bbc-8984-863b03abbca1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.136493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482de037-f818-4d9e-a8d1-1745e3df08d9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.153987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482de037-f818-4d9e-a8d1-1745e3df08d9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.157124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 893/1787] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:27.161487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec219cc-c451-4bbc-8984-863b03abbca1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.165488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ed4d7-bfee-4f72-a976-f4072563184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.166031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ed4d7-bfee-4f72-a976-f4072563184d admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.167251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ed4d7-bfee-4f72-a976-f4072563184d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.168047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ed4d7-bfee-4f72-a976-f4072563184d admin admin] PATH_INFO: `/v3/projects/e95169044859411e984841352bbf6dc6/users/2ded18b2e6764fb8b77fc4b625039324/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.173033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ed4d7-bfee-4f72-a976-f4072563184d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95169044859411e984841352bbf6dc6, user_id=2ded18b2e6764fb8b77fc4b625039324, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.176397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec219cc-c451-4bbc-8984-863b03abbca1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.187819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ed4d7-bfee-4f72-a976-f4072563184d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.204904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce7359-137e-4529-bdf7-65cdc80d3fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.207402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce7359-137e-4529-bdf7-65cdc80d3fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.208055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce7359-137e-4529-bdf7-65cdc80d3fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.208651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce7359-137e-4529-bdf7-65cdc80d3fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.212293 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce7359-137e-4529-bdf7-65cdc80d3fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.215091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 895/1788] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:27 2022] PUT /identity/v3/projects/e95169044859411e984841352bbf6dc6/users/2ded18b2e6764fb8b77fc4b625039324/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:27.216747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa802ae-8908-4852-a1e6-87a4dcd4d308 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.217047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa802ae-8908-4852-a1e6-87a4dcd4d308 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.217367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa802ae-8908-4852-a1e6-87a4dcd4d308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.220347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce7359-137e-4529-bdf7-65cdc80d3fe6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.256640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0fa802ae-8908-4852-a1e6-87a4dcd4d308 None None] MFA Rules not processed for user `2ded18b2e6764fb8b77fc4b625039324`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:27.258041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0fa802ae-8908-4852-a1e6-87a4dcd4d308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:27.262646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:27.327333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 894/1789] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:27 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:27.332874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce7359-137e-4529-bdf7-65cdc80d3fe6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.376767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 896/1790] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:27.385216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa802ae-8908-4852-a1e6-87a4dcd4d308 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.387316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0fa802ae-8908-4852-a1e6-87a4dcd4d308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:27.437618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce7359-137e-4529-bdf7-65cdc80d3fe6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.443270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417fdf7-744e-4d9b-b4fe-d487c0222916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.444048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417fdf7-744e-4d9b-b4fe-d487c0222916 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.444913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417fdf7-744e-4d9b-b4fe-d487c0222916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.444913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417fdf7-744e-4d9b-b4fe-d487c0222916 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.445710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417fdf7-744e-4d9b-b4fe-d487c0222916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.447791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417fdf7-744e-4d9b-b4fe-d487c0222916 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.473817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 895/1791] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:27 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:27.473817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417fdf7-744e-4d9b-b4fe-d487c0222916 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.477638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9417fdf7-744e-4d9b-b4fe-d487c0222916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:27.478648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa802ae-8908-4852-a1e6-87a4dcd4d308 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.483450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.484432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.484648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.484924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] PATH_INFO: `/v3/projects/77cfe7e01013404ca8b0ac9432ed28e6` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.486410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77cfe7e01013404ca8b0ac9432ed28e6)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.488365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.542707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77cfe7e01013404ca8b0ac9432ed28e6', 'request_id': 'req-6cd89498-68c7-47fb-aa1b-8f293777e5d0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:27.543827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417fdf7-744e-4d9b-b4fe-d487c0222916 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.549931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77cfe7e01013404ca8b0ac9432ed28e6', 'request_id': 'req-6cd89498-68c7-47fb-aa1b-8f293777e5d0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:27.559437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77cfe7e01013404ca8b0ac9432ed28e6', 'request_id': 'req-6cd89498-68c7-47fb-aa1b-8f293777e5d0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:27.572291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] The token cache is being invalidate because project 77cfe7e01013404ca8b0ac9432ed28e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:27.572600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:27.573328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:27.575265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 897/1792] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:27 2022] DELETE /identity/v3/projects/77cfe7e01013404ca8b0ac9432ed28e6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:27.581337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.581337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:27.617923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:27.623682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:27.643940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744aa78d-b2a7-4d33-a9cf-84350fdaed68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.644956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744aa78d-b2a7-4d33-a9cf-84350fdaed68 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.645401 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744aa78d-b2a7-4d33-a9cf-84350fdaed68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.645791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744aa78d-b2a7-4d33-a9cf-84350fdaed68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.648173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-744aa78d-b2a7-4d33-a9cf-84350fdaed68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:27.664590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd89498-68c7-47fb-aa1b-8f293777e5d0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.670450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51aaee-316d-412b-bc60-3f9489026e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.671468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51aaee-316d-412b-bc60-3f9489026e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.671763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51aaee-316d-412b-bc60-3f9489026e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.672045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51aaee-316d-412b-bc60-3f9489026e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.673424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51aaee-316d-412b-bc60-3f9489026e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.676403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51aaee-316d-412b-bc60-3f9489026e83 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.702108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 898/1793] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:27.704636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c458ba-6047-4d8c-8c18-064c765d007e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.705165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c458ba-6047-4d8c-8c18-064c765d007e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.705672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c458ba-6047-4d8c-8c18-064c765d007e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.741112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744aa78d-b2a7-4d33-a9cf-84350fdaed68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.742168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c7c458ba-6047-4d8c-8c18-064c765d007e None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:27.743119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744aa78d-b2a7-4d33-a9cf-84350fdaed68 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.745200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c7c458ba-6047-4d8c-8c18-064c765d007e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:27.755502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 896/1794] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:27.756598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744aa78d-b2a7-4d33-a9cf-84350fdaed68 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.771211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 899/1795] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:27.775596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf26c9b-6212-483f-9cd7-5e308bc02813 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.776390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf26c9b-6212-483f-9cd7-5e308bc02813 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.776518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744aa78d-b2a7-4d33-a9cf-84350fdaed68 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.777037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf26c9b-6212-483f-9cd7-5e308bc02813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.782796 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c46cc-cecf-4748-99c0-0f56323537b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.783840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c46cc-cecf-4748-99c0-0f56323537b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.784207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c46cc-cecf-4748-99c0-0f56323537b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.784751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c46cc-cecf-4748-99c0-0f56323537b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.786040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c46cc-cecf-4748-99c0-0f56323537b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.789043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c46cc-cecf-4748-99c0-0f56323537b7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.805389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 897/1796] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:27.808284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c46cc-cecf-4748-99c0-0f56323537b7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.815005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-bdf26c9b-6212-483f-9cd7-5e308bc02813 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:27.817596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bdf26c9b-6212-483f-9cd7-5e308bc02813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:27.834762 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c46cc-cecf-4748-99c0-0f56323537b7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.840951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e2428-6c16-4a83-99ba-e0b34b56a5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.841909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2428-6c16-4a83-99ba-e0b34b56a5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.842298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2428-6c16-4a83-99ba-e0b34b56a5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.842464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2428-6c16-4a83-99ba-e0b34b56a5bb admin admin] PATH_INFO: `/v3/projects/275b43d29f32455c9bcc3bae907d899d/users/e167d21227154c2083e51d50f89e66ec/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.844096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 900/1797] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:27.849805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf26c9b-6212-483f-9cd7-5e308bc02813 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.869045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf26c9b-6212-483f-9cd7-5e308bc02813 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.872735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e2428-6c16-4a83-99ba-e0b34b56a5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=275b43d29f32455c9bcc3bae907d899d, user_id=e167d21227154c2083e51d50f89e66ec, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.876183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e2428-6c16-4a83-99ba-e0b34b56a5bb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.877778 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cd6fb-2b48-4542-a162-bafc718d0c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:27.879138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cd6fb-2b48-4542-a162-bafc718d0c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.879707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cd6fb-2b48-4542-a162-bafc718d0c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.880262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cd6fb-2b48-4542-a162-bafc718d0c8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.881898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cd6fb-2b48-4542-a162-bafc718d0c8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:27.882472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cd6fb-2b48-4542-a162-bafc718d0c8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:27.887129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cd6fb-2b48-4542-a162-bafc718d0c8b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:27.902232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 898/1798] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:27 2022] PUT /identity/v3/projects/275b43d29f32455c9bcc3bae907d899d/users/e167d21227154c2083e51d50f89e66ec/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:27.913598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b9767-f4c5-48b1-966e-5e25f52a09f0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:27.913851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 901/1799] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:03:27.914482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b9767-f4c5-48b1-966e-5e25f52a09f0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:27.915224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b9767-f4c5-48b1-966e-5e25f52a09f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:27.921786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cd6fb-2b48-4542-a162-bafc718d0c8b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:27.943819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cd6fb-2b48-4542-a162-bafc718d0c8b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:27.952129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-154b9767-f4c5-48b1-966e-5e25f52a09f0 None None] MFA Rules not processed for user `e167d21227154c2083e51d50f89e66ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:27.954225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-154b9767-f4c5-48b1-966e-5e25f52a09f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:28.073642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c4273-1306-4f25-8e08-3905b69d4eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.075229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c4273-1306-4f25-8e08-3905b69d4eef admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.076660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c4273-1306-4f25-8e08-3905b69d4eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.077257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c4273-1306-4f25-8e08-3905b69d4eef admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.078339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 899/1800] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:28.079335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b9767-f4c5-48b1-966e-5e25f52a09f0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.079649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c4273-1306-4f25-8e08-3905b69d4eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.086350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c4273-1306-4f25-8e08-3905b69d4eef admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.113410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b9767-f4c5-48b1-966e-5e25f52a09f0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.125456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cc467-4cca-4c9a-bcbc-4f52b4f79150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.126498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cc467-4cca-4c9a-bcbc-4f52b4f79150 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.127526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cc467-4cca-4c9a-bcbc-4f52b4f79150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.127975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cc467-4cca-4c9a-bcbc-4f52b4f79150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.128889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845cc467-4cca-4c9a-bcbc-4f52b4f79150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.131412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845cc467-4cca-4c9a-bcbc-4f52b4f79150 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.150635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 902/1801] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:27 2022] POST /identity/v3/projects => generated 405 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:28.155087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c4273-1306-4f25-8e08-3905b69d4eef admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.166113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 900/1802] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:28 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:28.170845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cc467-4cca-4c9a-bcbc-4f52b4f79150 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.207780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:28.243389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c4273-1306-4f25-8e08-3905b69d4eef admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.249193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429dcfb-37f5-4863-9aaa-2ccbf874dfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.249682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cc467-4cca-4c9a-bcbc-4f52b4f79150 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.250064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429dcfb-37f5-4863-9aaa-2ccbf874dfeb admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.250422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429dcfb-37f5-4863-9aaa-2ccbf874dfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.250900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429dcfb-37f5-4863-9aaa-2ccbf874dfeb admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.252277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429dcfb-37f5-4863-9aaa-2ccbf874dfeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.254369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429dcfb-37f5-4863-9aaa-2ccbf874dfeb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.255464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ba3cb-da39-49ea-a3dc-f67023173111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.256353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ba3cb-da39-49ea-a3dc-f67023173111 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.256700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ba3cb-da39-49ea-a3dc-f67023173111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.257018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ba3cb-da39-49ea-a3dc-f67023173111 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.257810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ba3cb-da39-49ea-a3dc-f67023173111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.259864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ba3cb-da39-49ea-a3dc-f67023173111 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.281602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 903/1803] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:28 2022] POST /identity/v3/users => generated 303 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:28.283694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429dcfb-37f5-4863-9aaa-2ccbf874dfeb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.294569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429dcfb-37f5-4863-9aaa-2ccbf874dfeb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.298155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 901/1804] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:28 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:28.302378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ba3cb-da39-49ea-a3dc-f67023173111 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.308262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b112b2-45c6-4e6a-b1e9-86c91c1b0f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.309485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b112b2-45c6-4e6a-b1e9-86c91c1b0f25 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.309973 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b112b2-45c6-4e6a-b1e9-86c91c1b0f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.310482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b112b2-45c6-4e6a-b1e9-86c91c1b0f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.316190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ba3cb-da39-49ea-a3dc-f67023173111 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.321633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b112b2-45c6-4e6a-b1e9-86c91c1b0f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.324835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b112b2-45c6-4e6a-b1e9-86c91c1b0f25 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.335022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b800e0-9eed-4f5a-a83f-715786ac88fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.335712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b800e0-9eed-4f5a-a83f-715786ac88fd admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.336487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b800e0-9eed-4f5a-a83f-715786ac88fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.336487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b800e0-9eed-4f5a-a83f-715786ac88fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.337149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b800e0-9eed-4f5a-a83f-715786ac88fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.338970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b800e0-9eed-4f5a-a83f-715786ac88fd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.349186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 904/1805] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:28.352976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 902/1806] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:03:28.353752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b112b2-45c6-4e6a-b1e9-86c91c1b0f25 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.354452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b800e0-9eed-4f5a-a83f-715786ac88fd admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.363606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:28.366108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b800e0-9eed-4f5a-a83f-715786ac88fd admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.371570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd11c95-5855-4fcd-a080-201bbac685fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.372405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd11c95-5855-4fcd-a080-201bbac685fc admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.372769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd11c95-5855-4fcd-a080-201bbac685fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.373134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd11c95-5855-4fcd-a080-201bbac685fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.374333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd11c95-5855-4fcd-a080-201bbac685fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.376151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd11c95-5855-4fcd-a080-201bbac685fc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.398268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 903/1807] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:28.399404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd11c95-5855-4fcd-a080-201bbac685fc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.410472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd11c95-5855-4fcd-a080-201bbac685fc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.416936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a539b74-7766-4a9f-9da1-f93fb791b27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.418310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a539b74-7766-4a9f-9da1-f93fb791b27d admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.418310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a539b74-7766-4a9f-9da1-f93fb791b27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.418596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a539b74-7766-4a9f-9da1-f93fb791b27d admin admin] PATH_INFO: `/v3/projects/1934ef6f62b740eaba146d0a8db9c064/users/00c35b67e5d54dd1a95d84bc78e75c8f/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.441432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a539b74-7766-4a9f-9da1-f93fb791b27d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1934ef6f62b740eaba146d0a8db9c064, user_id=00c35b67e5d54dd1a95d84bc78e75c8f, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.444501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a539b74-7766-4a9f-9da1-f93fb791b27d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.468685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 904/1808] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:28 2022] PUT /identity/v3/projects/1934ef6f62b740eaba146d0a8db9c064/users/00c35b67e5d54dd1a95d84bc78e75c8f/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:28.471361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a539b74-7766-4a9f-9da1-f93fb791b27d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.473322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b112b2-45c6-4e6a-b1e9-86c91c1b0f25 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.555792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d251067-0e4a-4783-b424-bfac6699bedb tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ded18b2e6764fb8b77fc4b625039324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e95169044859411e984841352bbf6dc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.556966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d251067-0e4a-4783-b424-bfac6699bedb tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.557514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d251067-0e4a-4783-b424-bfac6699bedb tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.558017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d251067-0e4a-4783-b424-bfac6699bedb tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.559410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d251067-0e4a-4783-b424-bfac6699bedb tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:28.559968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d251067-0e4a-4783-b424-bfac6699bedb tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.563183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d251067-0e4a-4783-b424-bfac6699bedb tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.579028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a539b74-7766-4a9f-9da1-f93fb791b27d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.585388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104cd86-b980-4403-b1f1-206e452f7fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.588963 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 905/1809] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:28.589148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104cd86-b980-4403-b1f1-206e452f7fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.589148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104cd86-b980-4403-b1f1-206e452f7fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.589148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104cd86-b980-4403-b1f1-206e452f7fd4 admin admin] PATH_INFO: `/v3/projects/bc07c676ad5f44c18f24f0760c5639e6/users/afc18837ce8f4f6bacb2b37553a3e965/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.589413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d251067-0e4a-4783-b424-bfac6699bedb tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.603798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d251067-0e4a-4783-b424-bfac6699bedb tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.608696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a814dde-e2cb-48b7-9125-0d79c17de45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.609484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a814dde-e2cb-48b7-9125-0d79c17de45f admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.609676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104cd86-b980-4403-b1f1-206e452f7fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc07c676ad5f44c18f24f0760c5639e6, user_id=afc18837ce8f4f6bacb2b37553a3e965, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.609813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a814dde-e2cb-48b7-9125-0d79c17de45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.610049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a814dde-e2cb-48b7-9125-0d79c17de45f admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.610858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a814dde-e2cb-48b7-9125-0d79c17de45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.613132 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a814dde-e2cb-48b7-9125-0d79c17de45f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.615387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104cd86-b980-4403-b1f1-206e452f7fd4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.626128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 906/1810] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:28.627819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a814dde-e2cb-48b7-9125-0d79c17de45f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.637357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 905/1811] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:28 2022] PUT /identity/v3/projects/bc07c676ad5f44c18f24f0760c5639e6/users/afc18837ce8f4f6bacb2b37553a3e965/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:28.638373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104cd86-b980-4403-b1f1-206e452f7fd4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:28.658452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a814dde-e2cb-48b7-9125-0d79c17de45f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.701308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:28.720123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104cd86-b980-4403-b1f1-206e452f7fd4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:28.727089 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6e14f-bf85-4b44-93c8-d1d74cd23412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.728151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6e14f-bf85-4b44-93c8-d1d74cd23412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.728618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6e14f-bf85-4b44-93c8-d1d74cd23412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.729037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6e14f-bf85-4b44-93c8-d1d74cd23412 admin admin] PATH_INFO: `/v3/projects/1934ef6f62b740eaba146d0a8db9c064/users/00c35b67e5d54dd1a95d84bc78e75c8f/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.731344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd5f6e-3bd7-4d5f-81cb-72ab340ee596 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ded18b2e6764fb8b77fc4b625039324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e95169044859411e984841352bbf6dc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:28.731579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6e14f-bf85-4b44-93c8-d1d74cd23412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1934ef6f62b740eaba146d0a8db9c064, user_id=00c35b67e5d54dd1a95d84bc78e75c8f, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.732179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd5f6e-3bd7-4d5f-81cb-72ab340ee596 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.732533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd5f6e-3bd7-4d5f-81cb-72ab340ee596 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.732877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd5f6e-3bd7-4d5f-81cb-72ab340ee596 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.733734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd5f6e-3bd7-4d5f-81cb-72ab340ee596 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:28.735181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6e14f-bf85-4b44-93c8-d1d74cd23412 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.736174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd5f6e-3bd7-4d5f-81cb-72ab340ee596 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:28.748993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 906/1812] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:28 2022] PUT /identity/v3/projects/1934ef6f62b740eaba146d0a8db9c064/users/00c35b67e5d54dd1a95d84bc78e75c8f/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:28.751671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f54c6-c672-4af6-9e2b-4aeea52ec564 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.752730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f54c6-c672-4af6-9e2b-4aeea52ec564 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.753270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f54c6-c672-4af6-9e2b-4aeea52ec564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.771716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 907/1813] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:28 2022] POST /identity/v3/projects => generated 354 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:28.774555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ddcb7-5fdd-43c0-99cf-fe2750e24357 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:28.775536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ddcb7-5fdd-43c0-99cf-fe2750e24357 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:28.776278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ddcb7-5fdd-43c0-99cf-fe2750e24357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:28.813430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-095f54c6-c672-4af6-9e2b-4aeea52ec564 None None] MFA Rules not processed for user `afc18837ce8f4f6bacb2b37553a3e965`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:28.815402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-095f54c6-c672-4af6-9e2b-4aeea52ec564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:28.821689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b10ddcb7-5fdd-43c0-99cf-fe2750e24357 None None] MFA Rules not processed for user `00c35b67e5d54dd1a95d84bc78e75c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:28.823634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b10ddcb7-5fdd-43c0-99cf-fe2750e24357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:28.840505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:28.995097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 907/1814] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:28.997160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f54c6-c672-4af6-9e2b-4aeea52ec564 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:29.170350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f54c6-c672-4af6-9e2b-4aeea52ec564 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:29.184416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ded18b2e6764fb8b77fc4b625039324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e95169044859411e984841352bbf6dc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:29.185461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:29.185902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:29.186429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:29.187580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:29.188536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:29.191283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:29.201346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 908/1815] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:29.203500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ddcb7-5fdd-43c0-99cf-fe2750e24357 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:29.219407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 908/1816] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:29.239451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:29.246229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:29.357583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ddcb7-5fdd-43c0-99cf-fe2750e24357 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:29.368196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9831a5-c23b-4580-a3a5-a3a86cbdfea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:29.369446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9831a5-c23b-4580-a3a5-a3a86cbdfea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:29.370100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9831a5-c23b-4580-a3a5-a3a86cbdfea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:29.370785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9831a5-c23b-4580-a3a5-a3a86cbdfea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:29.372123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9831a5-c23b-4580-a3a5-a3a86cbdfea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:29.375777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9831a5-c23b-4580-a3a5-a3a86cbdfea1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:29.426949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fa664-264f-42bc-bd0b-f49a50bb104e tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:29.434765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2b641-d8f0-494d-80dd-b86c1f27de51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:29.436526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2b641-d8f0-494d-80dd-b86c1f27de51 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:29.437301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2b641-d8f0-494d-80dd-b86c1f27de51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:29.438249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2b641-d8f0-494d-80dd-b86c1f27de51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:29.443866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 909/1817] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:29 2022] POST /identity/v3/projects => generated 409 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:29.445871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9831a5-c23b-4580-a3a5-a3a86cbdfea1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:29.459426 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2b641-d8f0-494d-80dd-b86c1f27de51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:29.462012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2b641-d8f0-494d-80dd-b86c1f27de51 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:29.480350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 909/1818] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:29.481786 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2b641-d8f0-494d-80dd-b86c1f27de51 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:29.501158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2b641-d8f0-494d-80dd-b86c1f27de51 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:29.517904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:29.634151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9831a5-c23b-4580-a3a5-a3a86cbdfea1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:29.658842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ded18b2e6764fb8b77fc4b625039324', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e95169044859411e984841352bbf6dc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:29.661355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:29.661940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:29.662538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] PATH_INFO: `/v3/projects/122b463f650447f687c38f723f640efc` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:29.665395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Authorizing `identity:delete_project(project_id=122b463f650447f687c38f723f640efc)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:29.673304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:29.761432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:29.764812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a48c2-4d18-4225-b160-9b300bc43db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:29.769329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a48c2-4d18-4225-b160-9b300bc43db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:29.771291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a48c2-4d18-4225-b160-9b300bc43db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:29.771291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a48c2-4d18-4225-b160-9b300bc43db9 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:29.776404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a48c2-4d18-4225-b160-9b300bc43db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:29.784650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a48c2-4d18-4225-b160-9b300bc43db9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:29.806837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '122b463f650447f687c38f723f640efc', 'request_id': 'req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:29.822774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '122b463f650447f687c38f723f640efc', 'request_id': 'req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:29.836256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 910/1819] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:29 2022] POST /identity/v3/users => generated 311 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:29.837501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a48c2-4d18-4225-b160-9b300bc43db9 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:29.843357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '122b463f650447f687c38f723f640efc', 'request_id': 'req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:29.876374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] The token cache is being invalidate because project 122b463f650447f687c38f723f640efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:29.877471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:29.879065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:29.887753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 910/1820] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:29 2022] DELETE /identity/v3/projects/122b463f650447f687c38f723f640efc => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:29.887753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:29.888372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:29.960358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a48c2-4d18-4225-b160-9b300bc43db9 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:30.068714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:30.073185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.073185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.073185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:30.077210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:30.105811 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff83a-f2f3-41cc-a6f0-1fe0316c4589 tempest-VolumeQuotaClassesTest-505807588 tempest-VolumeQuotaClassesTest-505807588-project-admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:30.120031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009533e-61cd-4405-991d-607d246af6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:30.120468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009533e-61cd-4405-991d-607d246af6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.120898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009533e-61cd-4405-991d-607d246af6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.121252 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009533e-61cd-4405-991d-607d246af6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:30.122166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009533e-61cd-4405-991d-607d246af6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:30.124543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009533e-61cd-4405-991d-607d246af6b0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:30.151013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 911/1821] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:30.155466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009533e-61cd-4405-991d-607d246af6b0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:30.167820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009533e-61cd-4405-991d-607d246af6b0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:30.178156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab43f4-6c38-4958-9c7f-ee88d0833d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:30.179134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab43f4-6c38-4958-9c7f-ee88d0833d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.179507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab43f4-6c38-4958-9c7f-ee88d0833d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.179794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab43f4-6c38-4958-9c7f-ee88d0833d0d admin admin] PATH_INFO: `/v3/projects/caea30d3a01643cea82a4b897f5c6a4c/users/0f6d5c87d6194023bcd19c9bb7ca6344/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:30.205795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab43f4-6c38-4958-9c7f-ee88d0833d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=caea30d3a01643cea82a4b897f5c6a4c, user_id=0f6d5c87d6194023bcd19c9bb7ca6344, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:30.209875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab43f4-6c38-4958-9c7f-ee88d0833d0d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:30.220277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:30.222420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:30.227227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 912/1822] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:30 2022] PUT /identity/v3/projects/caea30d3a01643cea82a4b897f5c6a4c/users/0f6d5c87d6194023bcd19c9bb7ca6344/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:30.231579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab43f4-6c38-4958-9c7f-ee88d0833d0d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:30.241684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 911/1823] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:30.343477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:30.346267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:30.350879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab43f4-6c38-4958-9c7f-ee88d0833d0d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:30.366757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037262f-4e40-45a3-a88c-ec6194478523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:30.366757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037262f-4e40-45a3-a88c-ec6194478523 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.366757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037262f-4e40-45a3-a88c-ec6194478523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.366757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037262f-4e40-45a3-a88c-ec6194478523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:30.367332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037262f-4e40-45a3-a88c-ec6194478523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:30.367332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037262f-4e40-45a3-a88c-ec6194478523 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:30.374658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970d71f-843e-4d8b-84df-848b33b721d6 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:30.387789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:30.389628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.397490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.397490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] PATH_INFO: `/v3/users/81c6b704fb5b4719a89a7d752b211962` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:30.397490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c6b704fb5b4719a89a7d752b211962)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:30.397490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:30.412710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 913/1824] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:30.417822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037262f-4e40-45a3-a88c-ec6194478523 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:30.422335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:30.443614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037262f-4e40-45a3-a88c-ec6194478523 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:30.456115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6ce67-b6e9-4bfc-b88c-52704536d244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:30.458486 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6ce67-b6e9-4bfc-b88c-52704536d244 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.459000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6ce67-b6e9-4bfc-b88c-52704536d244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.459821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6ce67-b6e9-4bfc-b88c-52704536d244 admin admin] PATH_INFO: `/v3/projects/caea30d3a01643cea82a4b897f5c6a4c/users/0f6d5c87d6194023bcd19c9bb7ca6344/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:30.464961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6ce67-b6e9-4bfc-b88c-52704536d244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caea30d3a01643cea82a4b897f5c6a4c, user_id=0f6d5c87d6194023bcd19c9bb7ca6344, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:30.468362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6ce67-b6e9-4bfc-b88c-52704536d244 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:30.509689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 914/1825] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:30 2022] PUT /identity/v3/projects/caea30d3a01643cea82a4b897f5c6a4c/users/0f6d5c87d6194023bcd19c9bb7ca6344/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:30.509689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9eea61-3830-47b5-8d16-7e08245dd8bd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.509689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9eea61-3830-47b5-8d16-7e08245dd8bd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.509689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9eea61-3830-47b5-8d16-7e08245dd8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:30.538091 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c6b704fb5b4719a89a7d752b211962'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:30.572925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-aa9eea61-3830-47b5-8d16-7e08245dd8bd None None] MFA Rules not processed for user `0f6d5c87d6194023bcd19c9bb7ca6344`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:30.576420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c6b704fb5b4719a89a7d752b211962'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:30.577862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-aa9eea61-3830-47b5-8d16-7e08245dd8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:30.589174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:30.596676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c6b704fb5b4719a89a7d752b211962'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:30.598526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c6b704fb5b4719a89a7d752b211962'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:30.615738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3ecea6cb-da28-449d-8916-b358b7967fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c6b704fb5b4719a89a7d752b211962'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:30.637309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 912/1826] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:30 2022] DELETE /identity/v3/users/81c6b704fb5b4719a89a7d752b211962 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:30.644172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc7461-d9d0-4601-96be-806cfa46f2ba None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.644763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc7461-d9d0-4601-96be-806cfa46f2ba None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.645341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc7461-d9d0-4601-96be-806cfa46f2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:30.672748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b3bc7461-d9d0-4601-96be-806cfa46f2ba None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:30.675314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b3bc7461-d9d0-4601-96be-806cfa46f2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:30.809314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 915/1827] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:30 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:30.826813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9eea61-3830-47b5-8d16-7e08245dd8bd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:30.827382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-aa9eea61-3830-47b5-8d16-7e08245dd8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:30.841584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 913/1828] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:30.843582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc7461-d9d0-4601-96be-806cfa46f2ba None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:30.846742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b3bc7461-d9d0-4601-96be-806cfa46f2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:30.931141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9eea61-3830-47b5-8d16-7e08245dd8bd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:30.937473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:30.937713 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc7461-d9d0-4601-96be-806cfa46f2ba None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:30.938375 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.938720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.938936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] PATH_INFO: `/v3/users/e167d21227154c2083e51d50f89e66ec` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:30.940268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e167d21227154c2083e51d50f89e66ec)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:30.942715 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:30.943570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc5176-1979-48eb-b1e6-1881323e0958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:30.944233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5176-1979-48eb-b1e6-1881323e0958 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:30.944596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5176-1979-48eb-b1e6-1881323e0958 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:30.944788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5176-1979-48eb-b1e6-1881323e0958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:31.004858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:31.011008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e167d21227154c2083e51d50f89e66ec'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:31.042404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e167d21227154c2083e51d50f89e66ec'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:31.051608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc5176-1979-48eb-b1e6-1881323e0958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:31.053435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc5176-1979-48eb-b1e6-1881323e0958 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:31.055760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e167d21227154c2083e51d50f89e66ec'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:31.074000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e167d21227154c2083e51d50f89e66ec'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:31.075069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4e4307bf-42a6-4624-b301-22d82509328c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e167d21227154c2083e51d50f89e66ec'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:31.077539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 914/1829] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:30 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:31.087752 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc5176-1979-48eb-b1e6-1881323e0958 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:31.088217 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 916/1830] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:30 2022] DELETE /identity/v3/users/e167d21227154c2083e51d50f89e66ec => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:31.090388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-75dc5176-1979-48eb-b1e6-1881323e0958 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:31.091879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e852e-5ad3-40f0-95fd-af54cef1f38b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:31.092504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e852e-5ad3-40f0-95fd-af54cef1f38b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:31.092795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e852e-5ad3-40f0-95fd-af54cef1f38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:31.114992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-888e852e-5ad3-40f0-95fd-af54cef1f38b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:31.116021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-888e852e-5ad3-40f0-95fd-af54cef1f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:31.143875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:31.182143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc5176-1979-48eb-b1e6-1881323e0958 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:31.217580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 917/1831] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:31.229391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e852e-5ad3-40f0-95fd-af54cef1f38b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:31.232340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-888e852e-5ad3-40f0-95fd-af54cef1f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:31.271078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:31.271649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:31.271924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:31.272609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:31.274967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:31.292682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:31.294162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:31.310367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 915/1832] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:31.311362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:31.313015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:31.323717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e852e-5ad3-40f0-95fd-af54cef1f38b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:31.326431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41710903-9c3a-4075-ba7b-0168b9b9b9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:31.326645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41710903-9c3a-4075-ba7b-0168b9b9b9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:31.326915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41710903-9c3a-4075-ba7b-0168b9b9b9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:31.327290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41710903-9c3a-4075-ba7b-0168b9b9b9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:31.338786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41710903-9c3a-4075-ba7b-0168b9b9b9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:31.340331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41710903-9c3a-4075-ba7b-0168b9b9b9b9 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:31.353643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 918/1833] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:31.413187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93d5a-467d-4e14-a4b2-0239e0ead83c service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:31.417586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:31.418341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:31.418698 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:31.418984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:31.422054 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:31.511273 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:31.527975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:31.529415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:31.540985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 916/1834] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:31 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:31.842103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41710903-9c3a-4075-ba7b-0168b9b9b9b9 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:31.844002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:31.846148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:31.847396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-41710903-9c3a-4075-ba7b-0168b9b9b9b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:31.884130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9b2a9-d07a-40fc-ac36-af8a0265faad service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:31.885539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41710903-9c3a-4075-ba7b-0168b9b9b9b9 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:31.904154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:31.907498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:31.911573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:31.916621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:31.916915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:31.920863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] PATH_INFO: `/v3/projects/aa045b7f376740d7a8e6e4308075e4e7` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:31.921244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:31.921725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] PATH_INFO: `/v3/projects/275b43d29f32455c9bcc3bae907d899d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:31.924845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa045b7f376740d7a8e6e4308075e4e7)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:31.934059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:31.934277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=275b43d29f32455c9bcc3bae907d899d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:31.945412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:32.052527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa045b7f376740d7a8e6e4308075e4e7', 'request_id': 'req-de75299b-035f-4718-930c-2ee9c351e06f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.059131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa045b7f376740d7a8e6e4308075e4e7', 'request_id': 'req-de75299b-035f-4718-930c-2ee9c351e06f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.079674 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa045b7f376740d7a8e6e4308075e4e7', 'request_id': 'req-de75299b-035f-4718-930c-2ee9c351e06f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.095592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] The token cache is being invalidate because project aa045b7f376740d7a8e6e4308075e4e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:32.096567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.098560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.102155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 917/1835] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:31 2022] DELETE /identity/v3/projects/aa045b7f376740d7a8e6e4308075e4e7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:32.114974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:32.117371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:32.153817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275b43d29f32455c9bcc3bae907d899d', 'request_id': 'req-547e8140-413f-41e3-a30a-bef79b491045'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.161570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275b43d29f32455c9bcc3bae907d899d', 'request_id': 'req-547e8140-413f-41e3-a30a-bef79b491045'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.179324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275b43d29f32455c9bcc3bae907d899d', 'request_id': 'req-547e8140-413f-41e3-a30a-bef79b491045'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.209138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] The token cache is being invalidate because project 275b43d29f32455c9bcc3bae907d899d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:32.209138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.210868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.225660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 919/1836] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:31 2022] DELETE /identity/v3/projects/275b43d29f32455c9bcc3bae907d899d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:32.225660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:32.225660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:32.329273 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75299b-035f-4718-930c-2ee9c351e06f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:32.401598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:32.453857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:32.456294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:32.456652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:32.459398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] PATH_INFO: `/v3/users/2ded18b2e6764fb8b77fc4b625039324` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:32.464562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ded18b2e6764fb8b77fc4b625039324)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:32.466630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e8140-413f-41e3-a30a-bef79b491045 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:32.469481 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:32.482554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:32.484104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:32.484280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:32.484445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] PATH_INFO: `/v3/users/00c35b67e5d54dd1a95d84bc78e75c8f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:32.487191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c35b67e5d54dd1a95d84bc78e75c8f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:32.489665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:32.496237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:32.553398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ded18b2e6764fb8b77fc4b625039324'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.579843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ded18b2e6764fb8b77fc4b625039324'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.586362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c35b67e5d54dd1a95d84bc78e75c8f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.592122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ded18b2e6764fb8b77fc4b625039324'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.593257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ded18b2e6764fb8b77fc4b625039324'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.603753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ded18b2e6764fb8b77fc4b625039324'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.611765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 918/1837] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:32 2022] DELETE /identity/v3/users/2ded18b2e6764fb8b77fc4b625039324 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:32.621198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c35b67e5d54dd1a95d84bc78e75c8f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.634046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c35b67e5d54dd1a95d84bc78e75c8f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.652300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c35b67e5d54dd1a95d84bc78e75c8f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.652300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c35b67e5d54dd1a95d84bc78e75c8f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:32.653844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 920/1838] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:32 2022] DELETE /identity/v3/users/00c35b67e5d54dd1a95d84bc78e75c8f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:33.154308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:33.154914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:33.255184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:33.256857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:33.279570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f23f6-4483-4ec9-949e-0c0edabd9e7f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:33.451010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb11b7a-6d97-4181-bca5-787fb75aa390 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:33.468000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:33.474670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:33.474993 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:33.475280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] PATH_INFO: `/v3/projects/e95169044859411e984841352bbf6dc6` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:33.482246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95169044859411e984841352bbf6dc6)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:33.489124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:33.509025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:33.517638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:33.519485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:33.530963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] PATH_INFO: `/v3/projects/1934ef6f62b740eaba146d0a8db9c064` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:33.535475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1934ef6f62b740eaba146d0a8db9c064)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:33.540939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:33.635237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:33.639079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1934ef6f62b740eaba146d0a8db9c064', 'request_id': 'req-e662cae8-cc4b-44c5-807d-62b35784bb22'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.648611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95169044859411e984841352bbf6dc6', 'request_id': 'req-707954ae-2cba-450f-9369-2b43bb7135b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.649614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1934ef6f62b740eaba146d0a8db9c064', 'request_id': 'req-e662cae8-cc4b-44c5-807d-62b35784bb22'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.675117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95169044859411e984841352bbf6dc6', 'request_id': 'req-707954ae-2cba-450f-9369-2b43bb7135b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.680178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1934ef6f62b740eaba146d0a8db9c064', 'request_id': 'req-e662cae8-cc4b-44c5-807d-62b35784bb22'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.697930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95169044859411e984841352bbf6dc6', 'request_id': 'req-707954ae-2cba-450f-9369-2b43bb7135b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.706854 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] The token cache is being invalidate because project 1934ef6f62b740eaba146d0a8db9c064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:33.707154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.709285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e662cae8-cc4b-44c5-807d-62b35784bb22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.710038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:33.711403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 919/1839] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:33 2022] DELETE /identity/v3/projects/1934ef6f62b740eaba146d0a8db9c064 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:33.718330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0449e-cb50-4138-af8e-69b6d65cc639 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:33.718330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0449e-cb50-4138-af8e-69b6d65cc639 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:33.718610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0449e-cb50-4138-af8e-69b6d65cc639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:33.726207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] The token cache is being invalidate because project e95169044859411e984841352bbf6dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:33.728215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.729635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-707954ae-2cba-450f-9369-2b43bb7135b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:33.734337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 921/1840] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:33 2022] DELETE /identity/v3/projects/e95169044859411e984841352bbf6dc6 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:33.741465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbde7f-1f52-4659-a52f-746a339e9422 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:33.741746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbde7f-1f52-4659-a52f-746a339e9422 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:33.742078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbde7f-1f52-4659-a52f-746a339e9422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:33.771139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b0e0449e-cb50-4138-af8e-69b6d65cc639 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:33.772817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b0e0449e-cb50-4138-af8e-69b6d65cc639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:33.776886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5adbde7f-1f52-4659-a52f-746a339e9422 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:33.778307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5adbde7f-1f52-4659-a52f-746a339e9422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:33.897572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 922/1841] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:33.907819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a40d86-a565-4030-a9c7-18b2740abe44 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:33.908119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a40d86-a565-4030-a9c7-18b2740abe44 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:33.908314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a40d86-a565-4030-a9c7-18b2740abe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:33.937331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f7a40d86-a565-4030-a9c7-18b2740abe44 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:33.939185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f7a40d86-a565-4030-a9c7-18b2740abe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:33.957278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 923/1842] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:33.958420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a40d86-a565-4030-a9c7-18b2740abe44 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:33.976260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 920/1843] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:33.982775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a40d86-a565-4030-a9c7-18b2740abe44 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:33.984945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed7e8a-3653-4178-91f7-997e68b7bb2c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:33.985341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed7e8a-3653-4178-91f7-997e68b7bb2c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:33.985557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed7e8a-3653-4178-91f7-997e68b7bb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.001331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594ab43-0bf8-4c30-aa10-7aa4e7b99bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.008935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594ab43-0bf8-4c30-aa10-7aa4e7b99bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.009254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594ab43-0bf8-4c30-aa10-7aa4e7b99bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.009580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594ab43-0bf8-4c30-aa10-7aa4e7b99bfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.010903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594ab43-0bf8-4c30-aa10-7aa4e7b99bfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:34.011453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594ab43-0bf8-4c30-aa10-7aa4e7b99bfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.019671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594ab43-0bf8-4c30-aa10-7aa4e7b99bfa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.043883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0eed7e8a-3653-4178-91f7-997e68b7bb2c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:34.046467 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0eed7e8a-3653-4178-91f7-997e68b7bb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:34.064559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 921/1844] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:34.072930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed7e8a-3653-4178-91f7-997e68b7bb2c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.089136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 924/1845] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:34.098607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed7e8a-3653-4178-91f7-997e68b7bb2c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.109301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de06fc-4b00-476e-9aab-96f8c0d480ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.110812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de06fc-4b00-476e-9aab-96f8c0d480ae admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.110979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594ab43-0bf8-4c30-aa10-7aa4e7b99bfa admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.114181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de06fc-4b00-476e-9aab-96f8c0d480ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.114756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de06fc-4b00-476e-9aab-96f8c0d480ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.116656 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de06fc-4b00-476e-9aab-96f8c0d480ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:34.117193 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de06fc-4b00-476e-9aab-96f8c0d480ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.120644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de06fc-4b00-476e-9aab-96f8c0d480ae admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.160731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 922/1846] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:34.164424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de06fc-4b00-476e-9aab-96f8c0d480ae admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.164682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594ab43-0bf8-4c30-aa10-7aa4e7b99bfa admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.176082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b19f8c-3d1d-4869-83de-7fd699eafea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.179326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b19f8c-3d1d-4869-83de-7fd699eafea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.180503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b19f8c-3d1d-4869-83de-7fd699eafea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.180980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b19f8c-3d1d-4869-83de-7fd699eafea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.182245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b19f8c-3d1d-4869-83de-7fd699eafea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.185083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b19f8c-3d1d-4869-83de-7fd699eafea1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.185760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de06fc-4b00-476e-9aab-96f8c0d480ae admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.192442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3553d-992f-487a-b9e1-4a196679e1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.193220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3553d-992f-487a-b9e1-4a196679e1fb admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.193583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3553d-992f-487a-b9e1-4a196679e1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.193931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3553d-992f-487a-b9e1-4a196679e1fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.194757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3553d-992f-487a-b9e1-4a196679e1fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.197122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3553d-992f-487a-b9e1-4a196679e1fb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.233979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 925/1847] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:34 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:34.240092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b19f8c-3d1d-4869-83de-7fd699eafea1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.250505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 923/1848] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:34 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:34.252502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3553d-992f-487a-b9e1-4a196679e1fb admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.378553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b19f8c-3d1d-4869-83de-7fd699eafea1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.404624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3553d-992f-487a-b9e1-4a196679e1fb admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.425000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715f59d-fd93-4515-ad5d-ca3cfc62cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.428621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715f59d-fd93-4515-ad5d-ca3cfc62cfde admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.429369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715f59d-fd93-4515-ad5d-ca3cfc62cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.429663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715f59d-fd93-4515-ad5d-ca3cfc62cfde admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.431156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715f59d-fd93-4515-ad5d-ca3cfc62cfde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.433332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715f59d-fd93-4515-ad5d-ca3cfc62cfde admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.470115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 924/1849] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:34 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:34.476591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715f59d-fd93-4515-ad5d-ca3cfc62cfde admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.496944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715f59d-fd93-4515-ad5d-ca3cfc62cfde admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.505289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3250-ebb1-474d-9c1d-b9c81d6d1ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.506227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3250-ebb1-474d-9c1d-b9c81d6d1ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.506516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3250-ebb1-474d-9c1d-b9c81d6d1ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.506830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b3250-ebb1-474d-9c1d-b9c81d6d1ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.507829 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b3250-ebb1-474d-9c1d-b9c81d6d1ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.509667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b3250-ebb1-474d-9c1d-b9c81d6d1ad5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.510454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2678110-e37d-4414-b1dd-aad4a0bcfe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.511873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2678110-e37d-4414-b1dd-aad4a0bcfe20 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.512380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2678110-e37d-4414-b1dd-aad4a0bcfe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.512827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2678110-e37d-4414-b1dd-aad4a0bcfe20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.514260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2678110-e37d-4414-b1dd-aad4a0bcfe20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.518830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:34.520232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2678110-e37d-4414-b1dd-aad4a0bcfe20 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.541354 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 925/1850] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:34.544818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 926/1851] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:34 2022] POST /identity/v3/users => generated 295 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:34.545197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2678110-e37d-4414-b1dd-aad4a0bcfe20 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.549627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3250-ebb1-474d-9c1d-b9c81d6d1ad5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.562671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b3250-ebb1-474d-9c1d-b9c81d6d1ad5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.564416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2678110-e37d-4414-b1dd-aad4a0bcfe20 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.570122 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bf01d-18d1-4c5a-8a4c-41f1491239dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.571074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bf01d-18d1-4c5a-8a4c-41f1491239dd admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.571568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bf01d-18d1-4c5a-8a4c-41f1491239dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.572312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bf01d-18d1-4c5a-8a4c-41f1491239dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.573577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bf01d-18d1-4c5a-8a4c-41f1491239dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.576439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bf01d-18d1-4c5a-8a4c-41f1491239dd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.581437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c677a4-9400-4c42-a3c7-403aedfcdead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.582123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c677a4-9400-4c42-a3c7-403aedfcdead admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.582413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c677a4-9400-4c42-a3c7-403aedfcdead admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.583228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c677a4-9400-4c42-a3c7-403aedfcdead admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.583974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c677a4-9400-4c42-a3c7-403aedfcdead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.585632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c677a4-9400-4c42-a3c7-403aedfcdead admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.587817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 927/1852] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:03:34.592110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bf01d-18d1-4c5a-8a4c-41f1491239dd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.593210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:34.603953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bf01d-18d1-4c5a-8a4c-41f1491239dd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.604222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 926/1853] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:34.608873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c677a4-9400-4c42-a3c7-403aedfcdead admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.611289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573a789-1f74-4858-81b6-ee1239d6a6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.612049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573a789-1f74-4858-81b6-ee1239d6a6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.612388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573a789-1f74-4858-81b6-ee1239d6a6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.612744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573a789-1f74-4858-81b6-ee1239d6a6f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.613904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573a789-1f74-4858-81b6-ee1239d6a6f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.615621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573a789-1f74-4858-81b6-ee1239d6a6f6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.623583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c677a4-9400-4c42-a3c7-403aedfcdead admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.632091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 928/1854] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:03:34.632283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a727de-259a-47bd-8a8a-ce4c1e07415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.632972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a727de-259a-47bd-8a8a-ce4c1e07415d admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.633424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a727de-259a-47bd-8a8a-ce4c1e07415d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.633564 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a727de-259a-47bd-8a8a-ce4c1e07415d admin admin] PATH_INFO: `/v3/projects/1146e196c7d64f1ab2796c9f2893778c/users/f02a9ef0e7014bf0bebe2e05254240e4/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.633800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573a789-1f74-4858-81b6-ee1239d6a6f6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.644820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573a789-1f74-4858-81b6-ee1239d6a6f6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.649704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325e166-c8a0-43d2-901f-8b00924b99c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.650623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325e166-c8a0-43d2-901f-8b00924b99c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.650880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325e166-c8a0-43d2-901f-8b00924b99c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.651042 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a727de-259a-47bd-8a8a-ce4c1e07415d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1146e196c7d64f1ab2796c9f2893778c, user_id=f02a9ef0e7014bf0bebe2e05254240e4, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.651205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325e166-c8a0-43d2-901f-8b00924b99c7 admin admin] PATH_INFO: `/v3/projects/81d71a5a8bca49ba8c40c3532bd25dc4/users/fc57089ce21849fe8fecdf0c6758f298/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.653719 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a727de-259a-47bd-8a8a-ce4c1e07415d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.669500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325e166-c8a0-43d2-901f-8b00924b99c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d71a5a8bca49ba8c40c3532bd25dc4, user_id=fc57089ce21849fe8fecdf0c6758f298, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.672116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325e166-c8a0-43d2-901f-8b00924b99c7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.673002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 927/1855] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:34 2022] PUT /identity/v3/projects/1146e196c7d64f1ab2796c9f2893778c/users/f02a9ef0e7014bf0bebe2e05254240e4/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:34.688022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b5f40-2324-4075-a7dc-aef6dd03fc1a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.688398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 929/1856] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:34 2022] PUT /identity/v3/projects/81d71a5a8bca49ba8c40c3532bd25dc4/users/fc57089ce21849fe8fecdf0c6758f298/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:34.688596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b5f40-2324-4075-a7dc-aef6dd03fc1a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.688865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b5f40-2324-4075-a7dc-aef6dd03fc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.700223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f193e-bb65-4e3d-9440-b677c4729dd0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.700521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f193e-bb65-4e3d-9440-b677c4729dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.700804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f193e-bb65-4e3d-9440-b677c4729dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.718357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ac5b5f40-2324-4075-a7dc-aef6dd03fc1a None None] MFA Rules not processed for user `f02a9ef0e7014bf0bebe2e05254240e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:34.719587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ac5b5f40-2324-4075-a7dc-aef6dd03fc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:34.733476 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-898f193e-bb65-4e3d-9440-b677c4729dd0 None None] MFA Rules not processed for user `fc57089ce21849fe8fecdf0c6758f298`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:34.736173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-898f193e-bb65-4e3d-9440-b677c4729dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:34.811986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 928/1857] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:34.824526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b5f40-2324-4075-a7dc-aef6dd03fc1a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.826250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ac5b5f40-2324-4075-a7dc-aef6dd03fc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:34.838824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 930/1858] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:34.845723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f193e-bb65-4e3d-9440-b677c4729dd0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.891482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b5f40-2324-4075-a7dc-aef6dd03fc1a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.896194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fac78e-e1f3-4639-91e4-6756107b0efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.896890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fac78e-e1f3-4639-91e4-6756107b0efc service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.897213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fac78e-e1f3-4639-91e4-6756107b0efc service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.897514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fac78e-e1f3-4639-91e4-6756107b0efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.908294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fac78e-e1f3-4639-91e4-6756107b0efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.909813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fac78e-e1f3-4639-91e4-6756107b0efc service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.920035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 929/1859] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:03:34.934901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f193e-bb65-4e3d-9440-b677c4729dd0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.948026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd593b5f-cce2-4244-855d-fa62753a36d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.948602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fac78e-e1f3-4639-91e4-6756107b0efc service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:34.949019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd593b5f-cce2-4244-855d-fa62753a36d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.949353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd593b5f-cce2-4244-855d-fa62753a36d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.949637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd593b5f-cce2-4244-855d-fa62753a36d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.950360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd593b5f-cce2-4244-855d-fa62753a36d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.952306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd593b5f-cce2-4244-855d-fa62753a36d2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.969495 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fac78e-e1f3-4639-91e4-6756107b0efc service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:34.980490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:34.981683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:34.982036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:34.982429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] PATH_INFO: `/v3/users/f02a9ef0e7014bf0bebe2e05254240e4` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:34.984474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f02a9ef0e7014bf0bebe2e05254240e4)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:34.987039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:34.992606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 931/1860] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:34 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:34.993381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd593b5f-cce2-4244-855d-fa62753a36d2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:35.023989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:35.049474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f02a9ef0e7014bf0bebe2e05254240e4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:35.062811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:35.065292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f02a9ef0e7014bf0bebe2e05254240e4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:35.078813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f02a9ef0e7014bf0bebe2e05254240e4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:35.081061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f02a9ef0e7014bf0bebe2e05254240e4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:35.090063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd593b5f-cce2-4244-855d-fa62753a36d2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:35.090335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-66f57d05-03ac-4fa3-b916-eb708e66f9ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f02a9ef0e7014bf0bebe2e05254240e4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:35.098221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 930/1861] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:34 2022] DELETE /identity/v3/users/f02a9ef0e7014bf0bebe2e05254240e4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:35.099023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c593c-164e-45c7-a7b7-c31675e59c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:35.100140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c593c-164e-45c7-a7b7-c31675e59c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:35.100738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c593c-164e-45c7-a7b7-c31675e59c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:35.101564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c593c-164e-45c7-a7b7-c31675e59c0a admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:35.102333 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c593c-164e-45c7-a7b7-c31675e59c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:35.103167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eff8c-942b-49e5-972f-697a0be9619d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:35.103535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eff8c-942b-49e5-972f-697a0be9619d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:35.103887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eff8c-942b-49e5-972f-697a0be9619d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:35.105809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c593c-164e-45c7-a7b7-c31675e59c0a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:35.128151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-af8eff8c-942b-49e5-972f-697a0be9619d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:35.129420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-af8eff8c-942b-49e5-972f-697a0be9619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:35.140363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 932/1862] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:34 2022] POST /identity/v3/users => generated 301 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:35.145725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c593c-164e-45c7-a7b7-c31675e59c0a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:35.148784 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3c8c593c-164e-45c7-a7b7-c31675e59c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:35.173005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c593c-164e-45c7-a7b7-c31675e59c0a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:35.230007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 931/1863] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:35.241045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eff8c-942b-49e5-972f-697a0be9619d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:35.242838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-af8eff8c-942b-49e5-972f-697a0be9619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:35.268657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a916471-f895-404b-b807-6ade080f4cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:35.269388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a916471-f895-404b-b807-6ade080f4cff admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:35.269684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a916471-f895-404b-b807-6ade080f4cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:35.269965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a916471-f895-404b-b807-6ade080f4cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:35.270729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a916471-f895-404b-b807-6ade080f4cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:35.272420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a916471-f895-404b-b807-6ade080f4cff admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:35.285143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 933/1864] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:35.291182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a916471-f895-404b-b807-6ade080f4cff admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:35.302754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a916471-f895-404b-b807-6ade080f4cff admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:35.308327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d59caa-0bda-4f79-9e7a-5c8de2ae9866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:35.310091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d59caa-0bda-4f79-9e7a-5c8de2ae9866 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:35.310091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d59caa-0bda-4f79-9e7a-5c8de2ae9866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:35.310091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d59caa-0bda-4f79-9e7a-5c8de2ae9866 admin admin] PATH_INFO: `/v3/projects/fb08bbe47ba5422a9943e88f886c909d/users/def11fd251a24eee87e84951b64fcd6e/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:35.330817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d59caa-0bda-4f79-9e7a-5c8de2ae9866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb08bbe47ba5422a9943e88f886c909d, user_id=def11fd251a24eee87e84951b64fcd6e, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:35.333637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d59caa-0bda-4f79-9e7a-5c8de2ae9866 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:35.363977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 934/1865] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:35 2022] PUT /identity/v3/projects/fb08bbe47ba5422a9943e88f886c909d/users/def11fd251a24eee87e84951b64fcd6e/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:35.367455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eff8c-942b-49e5-972f-697a0be9619d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:35.369925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d59caa-0bda-4f79-9e7a-5c8de2ae9866 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:35.460477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897a858-16e9-47c7-8b18-abae24e04ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:35.462373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897a858-16e9-47c7-8b18-abae24e04ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:35.462836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897a858-16e9-47c7-8b18-abae24e04ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:35.463840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897a858-16e9-47c7-8b18-abae24e04ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:35.466701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d59caa-0bda-4f79-9e7a-5c8de2ae9866 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:35.474762 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61d4e1-a372-4eed-8bf0-7c7c34ce3f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:35.475614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897a858-16e9-47c7-8b18-abae24e04ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:35.475741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61d4e1-a372-4eed-8bf0-7c7c34ce3f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:35.475954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61d4e1-a372-4eed-8bf0-7c7c34ce3f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:35.476235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61d4e1-a372-4eed-8bf0-7c7c34ce3f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:35.477003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61d4e1-a372-4eed-8bf0-7c7c34ce3f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:35.477642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897a858-16e9-47c7-8b18-abae24e04ba2 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:35.478656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61d4e1-a372-4eed-8bf0-7c7c34ce3f35 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:35.499506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 935/1866] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:35.505907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61d4e1-a372-4eed-8bf0-7c7c34ce3f35 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:35.509118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 932/1867] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:03:35.522343 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61d4e1-a372-4eed-8bf0-7c7c34ce3f35 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:35.531900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcac6f9-d037-4fe1-afda-13de42570522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:35.533765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcac6f9-d037-4fe1-afda-13de42570522 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:35.534270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcac6f9-d037-4fe1-afda-13de42570522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:35.534612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcac6f9-d037-4fe1-afda-13de42570522 admin admin] PATH_INFO: `/v3/projects/fb08bbe47ba5422a9943e88f886c909d/users/def11fd251a24eee87e84951b64fcd6e/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:35.537768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcac6f9-d037-4fe1-afda-13de42570522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb08bbe47ba5422a9943e88f886c909d, user_id=def11fd251a24eee87e84951b64fcd6e, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:35.540321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcac6f9-d037-4fe1-afda-13de42570522 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:35.561191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 936/1868] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:35 2022] PUT /identity/v3/projects/fb08bbe47ba5422a9943e88f886c909d/users/def11fd251a24eee87e84951b64fcd6e/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:35.573545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506f099-c8d8-4ca8-a020-a2b2b99e14e8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:35.573895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506f099-c8d8-4ca8-a020-a2b2b99e14e8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:35.574183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506f099-c8d8-4ca8-a020-a2b2b99e14e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:35.660877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b506f099-c8d8-4ca8-a020-a2b2b99e14e8 None None] MFA Rules not processed for user `def11fd251a24eee87e84951b64fcd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:35.662192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b506f099-c8d8-4ca8-a020-a2b2b99e14e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:35.714957 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:35.805276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 933/1869] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:35.829457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcac6f9-d037-4fe1-afda-13de42570522 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:35.832848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9fcac6f9-d037-4fe1-afda-13de42570522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:35.847406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:35.966727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcac6f9-d037-4fe1-afda-13de42570522 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:35.972383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0b9d3-eb84-4698-be7e-9da7f550726f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:35.973309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0b9d3-eb84-4698-be7e-9da7f550726f service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:35.973591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0b9d3-eb84-4698-be7e-9da7f550726f service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:35.973881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0b9d3-eb84-4698-be7e-9da7f550726f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:35.984932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0b9d3-eb84-4698-be7e-9da7f550726f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:35.986514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0b9d3-eb84-4698-be7e-9da7f550726f service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:36.000175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 937/1870] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:36.032455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506f099-c8d8-4ca8-a020-a2b2b99e14e8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:36.038545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b506f099-c8d8-4ca8-a020-a2b2b99e14e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:36.179811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506f099-c8d8-4ca8-a020-a2b2b99e14e8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:36.188022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:36.188757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.189518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.190159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] PATH_INFO: `/v3/projects/1146e196c7d64f1ab2796c9f2893778c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.192961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1146e196c7d64f1ab2796c9f2893778c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:36.197349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:36.218100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0b9d3-eb84-4698-be7e-9da7f550726f service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:36.236005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0b9d3-eb84-4698-be7e-9da7f550726f service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:36.245065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:36.246283 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.246810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.247072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] PATH_INFO: `/v3/users/fc57089ce21849fe8fecdf0c6758f298` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.248663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc57089ce21849fe8fecdf0c6758f298)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:36.250961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:36.283565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1146e196c7d64f1ab2796c9f2893778c', 'request_id': 'req-3178c097-9f5d-4f82-ae10-e570122feae2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.292504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1146e196c7d64f1ab2796c9f2893778c', 'request_id': 'req-3178c097-9f5d-4f82-ae10-e570122feae2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.304142 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1146e196c7d64f1ab2796c9f2893778c', 'request_id': 'req-3178c097-9f5d-4f82-ae10-e570122feae2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.318422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc57089ce21849fe8fecdf0c6758f298'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.325347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] The token cache is being invalidate because project 1146e196c7d64f1ab2796c9f2893778c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:36.326178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.329830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3178c097-9f5d-4f82-ae10-e570122feae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.334525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 934/1871] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:36 2022] DELETE /identity/v3/projects/1146e196c7d64f1ab2796c9f2893778c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:36.341411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a49ad0-b0af-4994-abbc-3c2dedda38b8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.341791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a49ad0-b0af-4994-abbc-3c2dedda38b8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.342150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a49ad0-b0af-4994-abbc-3c2dedda38b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.342362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc57089ce21849fe8fecdf0c6758f298'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.354182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc57089ce21849fe8fecdf0c6758f298'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.365439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-95a49ad0-b0af-4994-abbc-3c2dedda38b8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:36.366730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-95a49ad0-b0af-4994-abbc-3c2dedda38b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:36.368598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc57089ce21849fe8fecdf0c6758f298'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.369894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-77919548-d962-4966-9964-28bd421a2678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc57089ce21849fe8fecdf0c6758f298'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:36.381380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 938/1872] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:36 2022] DELETE /identity/v3/users/fc57089ce21849fe8fecdf0c6758f298 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:36.386919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d1e12-bea2-4572-a1a0-5e0a49568a6c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.387415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d1e12-bea2-4572-a1a0-5e0a49568a6c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.387846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d1e12-bea2-4572-a1a0-5e0a49568a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.407866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9a8d1e12-bea2-4572-a1a0-5e0a49568a6c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:36.409203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9a8d1e12-bea2-4572-a1a0-5e0a49568a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:36.449028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:36.477132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 935/1873] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:36.491800 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193553a-0721-45c8-a012-714063ac7720 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.492394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193553a-0721-45c8-a012-714063ac7720 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.492970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193553a-0721-45c8-a012-714063ac7720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.495875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 939/1874] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:36.503137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d1e12-bea2-4572-a1a0-5e0a49568a6c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:36.505159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9a8d1e12-bea2-4572-a1a0-5e0a49568a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:36.524095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3193553a-0721-45c8-a012-714063ac7720 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:36.525513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3193553a-0721-45c8-a012-714063ac7720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:36.542216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 936/1875] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:36.543277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193553a-0721-45c8-a012-714063ac7720 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:36.549523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:36.555963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193553a-0721-45c8-a012-714063ac7720 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:36.562240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da30cf8-1a28-40c0-962f-71efdfe9f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:36.566215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da30cf8-1a28-40c0-962f-71efdfe9f364 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.566549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da30cf8-1a28-40c0-962f-71efdfe9f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.566840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da30cf8-1a28-40c0-962f-71efdfe9f364 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.572151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da30cf8-1a28-40c0-962f-71efdfe9f364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:36.572845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da30cf8-1a28-40c0-962f-71efdfe9f364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:36.574821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da30cf8-1a28-40c0-962f-71efdfe9f364 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:36.601848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 937/1876] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:36.607263 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da30cf8-1a28-40c0-962f-71efdfe9f364 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:36.630916 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da30cf8-1a28-40c0-962f-71efdfe9f364 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:36.637375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2d548-8fb2-47ae-aa95-54fad1242c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:36.638289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2d548-8fb2-47ae-aa95-54fad1242c21 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.638671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2d548-8fb2-47ae-aa95-54fad1242c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.639039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2d548-8fb2-47ae-aa95-54fad1242c21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.639418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d1e12-bea2-4572-a1a0-5e0a49568a6c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:36.640162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2d548-8fb2-47ae-aa95-54fad1242c21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:36.643306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2d548-8fb2-47ae-aa95-54fad1242c21 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:36.646649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66c47e-babf-4e4d-9c7d-44c80d09eff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:36.647862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66c47e-babf-4e4d-9c7d-44c80d09eff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.648471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66c47e-babf-4e4d-9c7d-44c80d09eff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.649077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66c47e-babf-4e4d-9c7d-44c80d09eff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.667872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66c47e-babf-4e4d-9c7d-44c80d09eff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:36.671148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66c47e-babf-4e4d-9c7d-44c80d09eff8 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:36.682859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 938/1877] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:36 2022] POST /identity/v3/projects => generated 361 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:36.687831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2d548-8fb2-47ae-aa95-54fad1242c21 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:36.699401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 940/1878] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:03:36.789140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2d548-8fb2-47ae-aa95-54fad1242c21 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:36.798153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bdba9-4e5e-4cfc-b53c-79a6d3a21037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:36.800302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bdba9-4e5e-4cfc-b53c-79a6d3a21037 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.801017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bdba9-4e5e-4cfc-b53c-79a6d3a21037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.801563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bdba9-4e5e-4cfc-b53c-79a6d3a21037 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.802912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bdba9-4e5e-4cfc-b53c-79a6d3a21037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:36.807595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bdba9-4e5e-4cfc-b53c-79a6d3a21037 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:36.845197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 939/1879] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:36 2022] POST /identity/v3/users => generated 281 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:36.846791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66c47e-babf-4e4d-9c7d-44c80d09eff8 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:36.860532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66c47e-babf-4e4d-9c7d-44c80d09eff8 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:36.867976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cb67d-bec2-47d4-a494-687397540679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:36.868667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cb67d-bec2-47d4-a494-687397540679 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.868954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cb67d-bec2-47d4-a494-687397540679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.869273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cb67d-bec2-47d4-a494-687397540679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.870028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cb67d-bec2-47d4-a494-687397540679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:36.871891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cb67d-bec2-47d4-a494-687397540679 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:36.885595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 941/1880] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:03:36.890149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bdba9-4e5e-4cfc-b53c-79a6d3a21037 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:36.908680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bdba9-4e5e-4cfc-b53c-79a6d3a21037 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:36.917985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78b354-ae06-4e5b-a9cf-eaa5ba889a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:36.918827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78b354-ae06-4e5b-a9cf-eaa5ba889a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.919131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78b354-ae06-4e5b-a9cf-eaa5ba889a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.919822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78b354-ae06-4e5b-a9cf-eaa5ba889a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.920595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78b354-ae06-4e5b-a9cf-eaa5ba889a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:36.922457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78b354-ae06-4e5b-a9cf-eaa5ba889a32 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:36.937131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 940/1881] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:36.942515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cb67d-bec2-47d4-a494-687397540679 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:36.953572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cb67d-bec2-47d4-a494-687397540679 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:36.959527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f71c84-e92f-4569-8b1b-748189978e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:36.960480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f71c84-e92f-4569-8b1b-748189978e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:36.960770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f71c84-e92f-4569-8b1b-748189978e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:36.961981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f71c84-e92f-4569-8b1b-748189978e2f admin admin] PATH_INFO: `/v3/projects/e7b110d41d5f4580928375ee5a1a1ca1/users/5857466441c94d698880c5ca6e4428ee/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:36.980060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f71c84-e92f-4569-8b1b-748189978e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b110d41d5f4580928375ee5a1a1ca1, user_id=5857466441c94d698880c5ca6e4428ee, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:36.982799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f71c84-e92f-4569-8b1b-748189978e2f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:37.006214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 942/1882] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:36 2022] PUT /identity/v3/projects/e7b110d41d5f4580928375ee5a1a1ca1/users/5857466441c94d698880c5ca6e4428ee/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:37.017984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc661000-7b8e-4ca4-a4e2-1c1220d2312e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:37.018971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc661000-7b8e-4ca4-a4e2-1c1220d2312e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:37.019642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc661000-7b8e-4ca4-a4e2-1c1220d2312e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:37.068197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fc661000-7b8e-4ca4-a4e2-1c1220d2312e None None] MFA Rules not processed for user `5857466441c94d698880c5ca6e4428ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:37.070628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fc661000-7b8e-4ca4-a4e2-1c1220d2312e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:37.187121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 941/1883] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:37.258524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f71c84-e92f-4569-8b1b-748189978e2f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:37.261749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e9f71c84-e92f-4569-8b1b-748189978e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:37.274790 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc661000-7b8e-4ca4-a4e2-1c1220d2312e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:37.276112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fc661000-7b8e-4ca4-a4e2-1c1220d2312e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:37.336978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:37.383481 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc661000-7b8e-4ca4-a4e2-1c1220d2312e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:37.387753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615c7c3-9502-4c4c-b76e-d6170dad22a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:37.388620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615c7c3-9502-4c4c-b76e-d6170dad22a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:37.388997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615c7c3-9502-4c4c-b76e-d6170dad22a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:37.389419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615c7c3-9502-4c4c-b76e-d6170dad22a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:37.397880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f71c84-e92f-4569-8b1b-748189978e2f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:37.401236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615c7c3-9502-4c4c-b76e-d6170dad22a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:37.402738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615c7c3-9502-4c4c-b76e-d6170dad22a8 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:37.405589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:37.406987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:37.407461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:37.409034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] PATH_INFO: `/v3/projects/81d71a5a8bca49ba8c40c3532bd25dc4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:37.410687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d71a5a8bca49ba8c40c3532bd25dc4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:37.414120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:37.414302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 942/1884] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:03:37.429148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615c7c3-9502-4c4c-b76e-d6170dad22a8 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:37.441868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615c7c3-9502-4c4c-b76e-d6170dad22a8 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:37.446990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:37.447882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:37.448166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:37.448449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] PATH_INFO: `/v3/users/5857466441c94d698880c5ca6e4428ee` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:37.449781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5857466441c94d698880c5ca6e4428ee)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:37.451780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:37.482435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d71a5a8bca49ba8c40c3532bd25dc4', 'request_id': 'req-f73bbd5c-039c-4288-ac67-73c5f107a573'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.492600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d71a5a8bca49ba8c40c3532bd25dc4', 'request_id': 'req-f73bbd5c-039c-4288-ac67-73c5f107a573'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.503838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d71a5a8bca49ba8c40c3532bd25dc4', 'request_id': 'req-f73bbd5c-039c-4288-ac67-73c5f107a573'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.512876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5857466441c94d698880c5ca6e4428ee'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.525265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] The token cache is being invalidate because project 81d71a5a8bca49ba8c40c3532bd25dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:37.525783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.526938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.529229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5857466441c94d698880c5ca6e4428ee'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.529725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 943/1885] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:37 2022] DELETE /identity/v3/projects/81d71a5a8bca49ba8c40c3532bd25dc4 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:37.536279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:37.537815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:37.539065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5857466441c94d698880c5ca6e4428ee'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.539755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5857466441c94d698880c5ca6e4428ee'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.549956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-45d8d49e-2350-4b78-b90a-109add7870cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5857466441c94d698880c5ca6e4428ee'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.559151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 943/1886] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:37 2022] DELETE /identity/v3/users/5857466441c94d698880c5ca6e4428ee => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:37.563966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:37.567633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c398-13c8-4f9f-8343-d371eb27ae7b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:37.567900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c398-13c8-4f9f-8343-d371eb27ae7b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:37.568230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c398-13c8-4f9f-8343-d371eb27ae7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:37.597746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8313c398-13c8-4f9f-8343-d371eb27ae7b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:37.599885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8313c398-13c8-4f9f-8343-d371eb27ae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:37.628904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bbd5c-039c-4288-ac67-73c5f107a573 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:37.704680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:37.706379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:37.707152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:37.707650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] PATH_INFO: `/v3/users/def11fd251a24eee87e84951b64fcd6e` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:37.710192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=def11fd251a24eee87e84951b64fcd6e)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:37.712225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 944/1887] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:37.714133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:37.720330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c398-13c8-4f9f-8343-d371eb27ae7b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:37.725550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8313c398-13c8-4f9f-8343-d371eb27ae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:37.810355 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:37.814447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def11fd251a24eee87e84951b64fcd6e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.836670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c398-13c8-4f9f-8343-d371eb27ae7b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:37.843528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def11fd251a24eee87e84951b64fcd6e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.845051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164f6ce-1ed2-4240-bab7-44e20eb61361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:37.846720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164f6ce-1ed2-4240-bab7-44e20eb61361 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:37.847156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164f6ce-1ed2-4240-bab7-44e20eb61361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:37.847629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164f6ce-1ed2-4240-bab7-44e20eb61361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:37.865837 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164f6ce-1ed2-4240-bab7-44e20eb61361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:37.867660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def11fd251a24eee87e84951b64fcd6e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.868776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164f6ce-1ed2-4240-bab7-44e20eb61361 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:37.886791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def11fd251a24eee87e84951b64fcd6e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.887677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 945/1888] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:37.888235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def11fd251a24eee87e84951b64fcd6e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:37.907365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 944/1889] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:37 2022] DELETE /identity/v3/users/def11fd251a24eee87e84951b64fcd6e => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:38.400693 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164f6ce-1ed2-4240-bab7-44e20eb61361 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:38.403365 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9164f6ce-1ed2-4240-bab7-44e20eb61361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:38.452836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164f6ce-1ed2-4240-bab7-44e20eb61361 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:38.590116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:38.592511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:38.593395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:38.594649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:38.595154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:38.595564 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] PATH_INFO: `/v3/projects/e7b110d41d5f4580928375ee5a1a1ca1` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:38.598569 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b110d41d5f4580928375ee5a1a1ca1)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:38.602033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:38.622784 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e962b6-774d-41b8-b772-21add6f3c173 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:38.632458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:38.633948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:38.634433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:38.634906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] PATH_INFO: `/v3/projects/fb08bbe47ba5422a9943e88f886c909d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:38.637076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb08bbe47ba5422a9943e88f886c909d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:38.640807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:38.689476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b110d41d5f4580928375ee5a1a1ca1', 'request_id': 'req-a60febbc-2abe-48bd-b0ed-42571b3805b4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.700654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b110d41d5f4580928375ee5a1a1ca1', 'request_id': 'req-a60febbc-2abe-48bd-b0ed-42571b3805b4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.716917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b110d41d5f4580928375ee5a1a1ca1', 'request_id': 'req-a60febbc-2abe-48bd-b0ed-42571b3805b4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.735779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:38.738247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] The token cache is being invalidate because project e7b110d41d5f4580928375ee5a1a1ca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:38.739068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.740512 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a60febbc-2abe-48bd-b0ed-42571b3805b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.744686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 946/1890] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:38 2022] DELETE /identity/v3/projects/e7b110d41d5f4580928375ee5a1a1ca1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:38.754339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1804435-7929-46b0-a066-daebe06c723d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:38.754663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1804435-7929-46b0-a066-daebe06c723d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:38.755248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1804435-7929-46b0-a066-daebe06c723d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:38.759514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb08bbe47ba5422a9943e88f886c909d', 'request_id': 'req-9139f16b-d78e-4e02-ad15-d29ba7cc1140'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.775307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb08bbe47ba5422a9943e88f886c909d', 'request_id': 'req-9139f16b-d78e-4e02-ad15-d29ba7cc1140'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.787562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb08bbe47ba5422a9943e88f886c909d', 'request_id': 'req-9139f16b-d78e-4e02-ad15-d29ba7cc1140'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.794523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f1804435-7929-46b0-a066-daebe06c723d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:38.796439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f1804435-7929-46b0-a066-daebe06c723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:38.808922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] The token cache is being invalidate because project fb08bbe47ba5422a9943e88f886c909d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:38.809427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.810504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9139f16b-d78e-4e02-ad15-d29ba7cc1140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:38.813261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 945/1891] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:38 2022] DELETE /identity/v3/projects/fb08bbe47ba5422a9943e88f886c909d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:38.820983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567780c3-5bb7-4267-b11d-892aece2ab56 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:38.821486 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567780c3-5bb7-4267-b11d-892aece2ab56 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:38.821955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567780c3-5bb7-4267-b11d-892aece2ab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:38.852094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-567780c3-5bb7-4267-b11d-892aece2ab56 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:38.853937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-567780c3-5bb7-4267-b11d-892aece2ab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:38.896419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 947/1892] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:38.905590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8b15f-5e04-4256-b350-7afee6090e01 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:38.906062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8b15f-5e04-4256-b350-7afee6090e01 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:38.906556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8b15f-5e04-4256-b350-7afee6090e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:38.933762 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-20a8b15f-5e04-4256-b350-7afee6090e01 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:38.935346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-20a8b15f-5e04-4256-b350-7afee6090e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:38.947862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 948/1893] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:38.950070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 946/1894] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:38.955837 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8b15f-5e04-4256-b350-7afee6090e01 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:38.960247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3979ee-16c4-4b01-ac77-e4a837011a53 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:38.960698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3979ee-16c4-4b01-ac77-e4a837011a53 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:38.961161 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3979ee-16c4-4b01-ac77-e4a837011a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:38.962471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:38.966925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8b15f-5e04-4256-b350-7afee6090e01 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:38.977242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110d818-ec67-4a7e-bff9-da7d7981eff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:38.979097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110d818-ec67-4a7e-bff9-da7d7981eff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:38.979491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110d818-ec67-4a7e-bff9-da7d7981eff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:38.980147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110d818-ec67-4a7e-bff9-da7d7981eff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:38.981068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110d818-ec67-4a7e-bff9-da7d7981eff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:38.981642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110d818-ec67-4a7e-bff9-da7d7981eff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:38.983539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110d818-ec67-4a7e-bff9-da7d7981eff5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:38.987285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9b3979ee-16c4-4b01-ac77-e4a837011a53 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:38.988709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9b3979ee-16c4-4b01-ac77-e4a837011a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:38.997030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 949/1895] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:03:38.998789 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 947/1896] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:39.001498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110d818-ec67-4a7e-bff9-da7d7981eff5 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.003958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3979ee-16c4-4b01-ac77-e4a837011a53 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.012232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110d818-ec67-4a7e-bff9-da7d7981eff5 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.015318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3979ee-16c4-4b01-ac77-e4a837011a53 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.017884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a087ee-b706-463a-ac4b-d8ec18ebf23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.018591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a087ee-b706-463a-ac4b-d8ec18ebf23a admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.018870 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a087ee-b706-463a-ac4b-d8ec18ebf23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.019145 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a087ee-b706-463a-ac4b-d8ec18ebf23a admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.019847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a087ee-b706-463a-ac4b-d8ec18ebf23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.021809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a087ee-b706-463a-ac4b-d8ec18ebf23a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.022105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bb6df-28b1-4875-8841-9cd6abd798bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.022810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bb6df-28b1-4875-8841-9cd6abd798bf admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.023079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bb6df-28b1-4875-8841-9cd6abd798bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.023580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bb6df-28b1-4875-8841-9cd6abd798bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.024295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bb6df-28b1-4875-8841-9cd6abd798bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:39.024611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bb6df-28b1-4875-8841-9cd6abd798bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.026852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bb6df-28b1-4875-8841-9cd6abd798bf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.040123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 948/1897] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:39.044589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bb6df-28b1-4875-8841-9cd6abd798bf admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.058051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bb6df-28b1-4875-8841-9cd6abd798bf admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.062523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 950/1898] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:39 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:39.067698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32cd55-130a-456f-8819-1ddb7343c706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.068381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32cd55-130a-456f-8819-1ddb7343c706 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.068671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32cd55-130a-456f-8819-1ddb7343c706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.068870 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a087ee-b706-463a-ac4b-d8ec18ebf23a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.068979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32cd55-130a-456f-8819-1ddb7343c706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.069844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32cd55-130a-456f-8819-1ddb7343c706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.072036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32cd55-130a-456f-8819-1ddb7343c706 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.103058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 949/1899] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:39 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:39.107521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32cd55-130a-456f-8819-1ddb7343c706 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.197558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a087ee-b706-463a-ac4b-d8ec18ebf23a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.239820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32cd55-130a-456f-8819-1ddb7343c706 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.248729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82b333-8e36-4a6d-a6fd-26fe542475d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.249967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82b333-8e36-4a6d-a6fd-26fe542475d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.250471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82b333-8e36-4a6d-a6fd-26fe542475d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.250982 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82b333-8e36-4a6d-a6fd-26fe542475d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.252873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82b333-8e36-4a6d-a6fd-26fe542475d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.256701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82b333-8e36-4a6d-a6fd-26fe542475d4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.291610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa8fd4-3080-444b-ae39-53d607d8378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.292611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa8fd4-3080-444b-ae39-53d607d8378e admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.293049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa8fd4-3080-444b-ae39-53d607d8378e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.293493 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa8fd4-3080-444b-ae39-53d607d8378e admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.298870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 950/1900] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:39 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:39.299052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa8fd4-3080-444b-ae39-53d607d8378e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.300114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa8fd4-3080-444b-ae39-53d607d8378e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.302284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82b333-8e36-4a6d-a6fd-26fe542475d4 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.322943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82b333-8e36-4a6d-a6fd-26fe542475d4 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.333690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050372bf-921b-454a-8044-41f074662fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.347356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050372bf-921b-454a-8044-41f074662fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.349757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050372bf-921b-454a-8044-41f074662fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.350038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 951/1901] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:39 2022] POST /identity/v3/users => generated 288 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:39.350690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050372bf-921b-454a-8044-41f074662fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.361347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050372bf-921b-454a-8044-41f074662fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.361347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050372bf-921b-454a-8044-41f074662fc3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.364647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa8fd4-3080-444b-ae39-53d607d8378e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.387146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa8fd4-3080-444b-ae39-53d607d8378e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.394983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 951/1902] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:39.399224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed745ad-02a5-4812-8a22-0b2c0d634483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.400134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050372bf-921b-454a-8044-41f074662fc3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.400537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed745ad-02a5-4812-8a22-0b2c0d634483 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.401168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed745ad-02a5-4812-8a22-0b2c0d634483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.401772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed745ad-02a5-4812-8a22-0b2c0d634483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.403229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed745ad-02a5-4812-8a22-0b2c0d634483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.406653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed745ad-02a5-4812-8a22-0b2c0d634483 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.409177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:39.413934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050372bf-921b-454a-8044-41f074662fc3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.420568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ae654-c528-4c97-b418-8d48792e63ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.421598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ae654-c528-4c97-b418-8d48792e63ad admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.422795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ae654-c528-4c97-b418-8d48792e63ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.423371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ae654-c528-4c97-b418-8d48792e63ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.424315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ae654-c528-4c97-b418-8d48792e63ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.426610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ae654-c528-4c97-b418-8d48792e63ad admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.438151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 952/1903] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:39.439632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 952/1904] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:03:39.445762 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ae654-c528-4c97-b418-8d48792e63ad admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.446580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed745ad-02a5-4812-8a22-0b2c0d634483 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.459899 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ae654-c528-4c97-b418-8d48792e63ad admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.466470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d77abd-45eb-44ab-8372-d8b325b2cfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.467698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d77abd-45eb-44ab-8372-d8b325b2cfdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.468177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d77abd-45eb-44ab-8372-d8b325b2cfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.468647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d77abd-45eb-44ab-8372-d8b325b2cfdb admin admin] PATH_INFO: `/v3/projects/f0574a9b7bb24827a309629b0a24b1fd/users/bba06789d6a44f6ba31e4d368b55878b/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.469437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed745ad-02a5-4812-8a22-0b2c0d634483 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.479242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b904a4-a9f5-4dde-8785-c305397aa611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.480547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b904a4-a9f5-4dde-8785-c305397aa611 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.481408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b904a4-a9f5-4dde-8785-c305397aa611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.481952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b904a4-a9f5-4dde-8785-c305397aa611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.483594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b904a4-a9f5-4dde-8785-c305397aa611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.487320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b904a4-a9f5-4dde-8785-c305397aa611 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.492268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d77abd-45eb-44ab-8372-d8b325b2cfdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0574a9b7bb24827a309629b0a24b1fd, user_id=bba06789d6a44f6ba31e4d368b55878b, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.496222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d77abd-45eb-44ab-8372-d8b325b2cfdb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.510418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 953/1905] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:39.515009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b904a4-a9f5-4dde-8785-c305397aa611 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.519206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 953/1906] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:39 2022] PUT /identity/v3/projects/f0574a9b7bb24827a309629b0a24b1fd/users/bba06789d6a44f6ba31e4d368b55878b/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:39.542706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5ef6f-5f29-4091-8c1b-264deb7b21c2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.543492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5ef6f-5f29-4091-8c1b-264deb7b21c2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.546349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5ef6f-5f29-4091-8c1b-264deb7b21c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.546487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b904a4-a9f5-4dde-8785-c305397aa611 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:39.622823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a9e5ef6f-5f29-4091-8c1b-264deb7b21c2 None None] MFA Rules not processed for user `bba06789d6a44f6ba31e4d368b55878b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:39.624873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a9e5ef6f-5f29-4091-8c1b-264deb7b21c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:39.649319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:39.676315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d86a8-159e-49a2-a810-a78ba8423e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:39.677772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d86a8-159e-49a2-a810-a78ba8423e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.678172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d86a8-159e-49a2-a810-a78ba8423e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.678555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d86a8-159e-49a2-a810-a78ba8423e9c admin admin] PATH_INFO: `/v3/projects/e2484f487efa444b870f2d7d82cf9af0/users/73a501cc0a6445709533da2b17de8a54/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.702425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d86a8-159e-49a2-a810-a78ba8423e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2484f487efa444b870f2d7d82cf9af0, user_id=73a501cc0a6445709533da2b17de8a54, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:39.705830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d86a8-159e-49a2-a810-a78ba8423e9c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:39.727649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 954/1907] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:39 2022] PUT /identity/v3/projects/e2484f487efa444b870f2d7d82cf9af0/users/73a501cc0a6445709533da2b17de8a54/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:39.739339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679def-9e6b-4184-a6b5-7dc4e2bc7869 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:39.739629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679def-9e6b-4184-a6b5-7dc4e2bc7869 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:39.739912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679def-9e6b-4184-a6b5-7dc4e2bc7869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:39.775400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 954/1908] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:39.780578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-af679def-9e6b-4184-a6b5-7dc4e2bc7869 None None] MFA Rules not processed for user `73a501cc0a6445709533da2b17de8a54`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:39.781855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-af679def-9e6b-4184-a6b5-7dc4e2bc7869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:39.833527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5ef6f-5f29-4091-8c1b-264deb7b21c2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:39.835598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a9e5ef6f-5f29-4091-8c1b-264deb7b21c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:39.916547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 955/1909] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:39.983153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5ef6f-5f29-4091-8c1b-264deb7b21c2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:40.083822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ad279-da49-426f-aec0-53972b62db19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:40.097248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ad279-da49-426f-aec0-53972b62db19 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:40.097928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ad279-da49-426f-aec0-53972b62db19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:40.098541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ad279-da49-426f-aec0-53972b62db19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:40.175719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:40.261225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ad279-da49-426f-aec0-53972b62db19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:40.266191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ad279-da49-426f-aec0-53972b62db19 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:40.295482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 955/1910] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:40.386482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af679def-9e6b-4184-a6b5-7dc4e2bc7869 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:40.388079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-af679def-9e6b-4184-a6b5-7dc4e2bc7869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:40.404938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af679def-9e6b-4184-a6b5-7dc4e2bc7869 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:40.414087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868644b-40f8-4aa1-8a41-cdb0b6f5c88c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:40.421620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868644b-40f8-4aa1-8a41-cdb0b6f5c88c service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:40.421620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868644b-40f8-4aa1-8a41-cdb0b6f5c88c service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:40.421620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868644b-40f8-4aa1-8a41-cdb0b6f5c88c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:40.443946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868644b-40f8-4aa1-8a41-cdb0b6f5c88c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:40.445483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868644b-40f8-4aa1-8a41-cdb0b6f5c88c service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:40.458123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 956/1911] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:40.515294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2751bf-e175-40e1-aaa5-db84c7012969 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:40.515961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2751bf-e175-40e1-aaa5-db84c7012969 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:40.516437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2751bf-e175-40e1-aaa5-db84c7012969 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:40.518270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 956/1912] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 19:03:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:03:40.532230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef019-8ea0-4d7d-8856-2e39d219db02 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:40.532663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef019-8ea0-4d7d-8856-2e39d219db02 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:40.533366 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef019-8ea0-4d7d-8856-2e39d219db02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:40.593751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b48ef019-8ea0-4d7d-8856-2e39d219db02 None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:40.596155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b48ef019-8ea0-4d7d-8856-2e39d219db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:40.823329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 957/1913] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 19:03:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:41.217635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2751bf-e175-40e1-aaa5-db84c7012969 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:41.219387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7a2751bf-e175-40e1-aaa5-db84c7012969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:41.237865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2751bf-e175-40e1-aaa5-db84c7012969 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:41.243674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45503b7-5ee0-433f-bc4c-b27207e35150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:41.244329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45503b7-5ee0-433f-bc4c-b27207e35150 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:41.244612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45503b7-5ee0-433f-bc4c-b27207e35150 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:41.244891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45503b7-5ee0-433f-bc4c-b27207e35150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:41.254531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45503b7-5ee0-433f-bc4c-b27207e35150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:41.256131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45503b7-5ee0-433f-bc4c-b27207e35150 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:41.267709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 957/1914] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:46.707291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef019-8ea0-4d7d-8856-2e39d219db02 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:46.709724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b48ef019-8ea0-4d7d-8856-2e39d219db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:46.770711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:46.817658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef019-8ea0-4d7d-8856-2e39d219db02 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:46.833954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:46.836855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:46.837554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:46.838396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] PATH_INFO: `/v3/users/afc18837ce8f4f6bacb2b37553a3e965` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:46.844079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc18837ce8f4f6bacb2b37553a3e965)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:46.850217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:46.964734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc18837ce8f4f6bacb2b37553a3e965'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:47.024246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc18837ce8f4f6bacb2b37553a3e965'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:47.045694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc18837ce8f4f6bacb2b37553a3e965'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:47.046839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc18837ce8f4f6bacb2b37553a3e965'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:47.077465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc18837ce8f4f6bacb2b37553a3e965'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:47.100378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 958/1915] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:46 2022] DELETE /identity/v3/users/afc18837ce8f4f6bacb2b37553a3e965 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:47.103165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ddff9-49b1-4508-89be-838b4c40e288 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:47.104000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ddff9-49b1-4508-89be-838b4c40e288 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:47.104318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ddff9-49b1-4508-89be-838b4c40e288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:47.161931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-db9ddff9-49b1-4508-89be-838b4c40e288 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:47.164015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-db9ddff9-49b1-4508-89be-838b4c40e288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:47.316904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 958/1916] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:47.341136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:47.347064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:47.493687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71173e-c421-4425-95c8-08b91f1337d7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:47.497902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93831020-bf72-4b71-a4ce-7d249a22e8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:47.498576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93831020-bf72-4b71-a4ce-7d249a22e8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:47.498974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93831020-bf72-4b71-a4ce-7d249a22e8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:47.499284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93831020-bf72-4b71-a4ce-7d249a22e8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:47.513761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93831020-bf72-4b71-a4ce-7d249a22e8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:47.515378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93831020-bf72-4b71-a4ce-7d249a22e8e9 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:47.533527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 959/1917] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:48.020623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ddff9-49b1-4508-89be-838b4c40e288 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:48.022305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-db9ddff9-49b1-4508-89be-838b4c40e288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:48.038011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ddff9-49b1-4508-89be-838b4c40e288 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:48.044744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:48.045745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:48.046086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:48.046429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] PATH_INFO: `/v3/projects/bc07c676ad5f44c18f24f0760c5639e6` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:48.047864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc07c676ad5f44c18f24f0760c5639e6)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:48.050444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:48.108324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc07c676ad5f44c18f24f0760c5639e6', 'request_id': 'req-58c8728f-0cc6-4379-b11b-f5ad782c9edc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.115547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc07c676ad5f44c18f24f0760c5639e6', 'request_id': 'req-58c8728f-0cc6-4379-b11b-f5ad782c9edc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.124656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc07c676ad5f44c18f24f0760c5639e6', 'request_id': 'req-58c8728f-0cc6-4379-b11b-f5ad782c9edc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.137509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] The token cache is being invalidate because project bc07c676ad5f44c18f24f0760c5639e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:48.137804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.138510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.140152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 959/1918] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:48 2022] DELETE /identity/v3/projects/bc07c676ad5f44c18f24f0760c5639e6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:48.145199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93831020-bf72-4b71-a4ce-7d249a22e8e9 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:48.155953 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93831020-bf72-4b71-a4ce-7d249a22e8e9 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:48.224325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:48.233715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:48.235045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:48.235794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:48.236429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] PATH_INFO: `/v3/users/0f6d5c87d6194023bcd19c9bb7ca6344` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:48.238333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6d5c87d6194023bcd19c9bb7ca6344)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:48.240396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:48.298083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6d5c87d6194023bcd19c9bb7ca6344'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.311419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6d5c87d6194023bcd19c9bb7ca6344'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.321708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6d5c87d6194023bcd19c9bb7ca6344'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.322575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6d5c87d6194023bcd19c9bb7ca6344'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.334016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-51419151-cf17-4a0c-82bb-b8323ab11513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6d5c87d6194023bcd19c9bb7ca6344'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.341019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 960/1919] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:48 2022] DELETE /identity/v3/users/0f6d5c87d6194023bcd19c9bb7ca6344 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:48.669830 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:48.671988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:48.696055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8728f-0cc6-4379-b11b-f5ad782c9edc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:48.744859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:48.769526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:48.770415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:48.770698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:48.770982 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] PATH_INFO: `/v3/projects/caea30d3a01643cea82a4b897f5c6a4c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:48.772259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caea30d3a01643cea82a4b897f5c6a4c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:48.774248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:48.830469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caea30d3a01643cea82a4b897f5c6a4c', 'request_id': 'req-c2fda069-7f87-42ac-92ad-7347790e58e4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.837675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caea30d3a01643cea82a4b897f5c6a4c', 'request_id': 'req-c2fda069-7f87-42ac-92ad-7347790e58e4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.846917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caea30d3a01643cea82a4b897f5c6a4c', 'request_id': 'req-c2fda069-7f87-42ac-92ad-7347790e58e4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.863269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] The token cache is being invalidate because project caea30d3a01643cea82a4b897f5c6a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:48.863754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.864718 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2fda069-7f87-42ac-92ad-7347790e58e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:48.867619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 960/1920] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:48 2022] DELETE /identity/v3/projects/caea30d3a01643cea82a4b897f5c6a4c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:48.872602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ede1c-9ba9-4e9d-a8c1-d44b13e1e466 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:48.873340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ede1c-9ba9-4e9d-a8c1-d44b13e1e466 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:48.874034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ede1c-9ba9-4e9d-a8c1-d44b13e1e466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:48.905256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-009ede1c-9ba9-4e9d-a8c1-d44b13e1e466 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:48.907271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-009ede1c-9ba9-4e9d-a8c1-d44b13e1e466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:49.052720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 961/1921] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:49.065316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afd3f8-83e6-4361-8690-25212f0739d9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:49.065740 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afd3f8-83e6-4361-8690-25212f0739d9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:49.066388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afd3f8-83e6-4361-8690-25212f0739d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:49.096478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a9afd3f8-83e6-4361-8690-25212f0739d9 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:49.098783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a9afd3f8-83e6-4361-8690-25212f0739d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:49.113863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 961/1922] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:49.120425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ede1c-9ba9-4e9d-a8c1-d44b13e1e466 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:49.137669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ede1c-9ba9-4e9d-a8c1-d44b13e1e466 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:49.146247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49b9df-b6bb-4efd-be8d-8fe867accf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:49.147328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49b9df-b6bb-4efd-be8d-8fe867accf54 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:49.147814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49b9df-b6bb-4efd-be8d-8fe867accf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:49.148311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49b9df-b6bb-4efd-be8d-8fe867accf54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:49.149756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49b9df-b6bb-4efd-be8d-8fe867accf54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:49.150302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49b9df-b6bb-4efd-be8d-8fe867accf54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:49.153347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49b9df-b6bb-4efd-be8d-8fe867accf54 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:49.178763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 962/1923] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:49.184376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afd3f8-83e6-4361-8690-25212f0739d9 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:49.196107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afd3f8-83e6-4361-8690-25212f0739d9 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:49.202442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae644b-fa16-49b0-9a7c-e816e7d0db71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:49.203253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae644b-fa16-49b0-9a7c-e816e7d0db71 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:49.203620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae644b-fa16-49b0-9a7c-e816e7d0db71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:49.204018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae644b-fa16-49b0-9a7c-e816e7d0db71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:49.205126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae644b-fa16-49b0-9a7c-e816e7d0db71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:49.207892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae644b-fa16-49b0-9a7c-e816e7d0db71 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:49.250684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 962/1924] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:49 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:49.255350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49b9df-b6bb-4efd-be8d-8fe867accf54 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:49.267674 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49b9df-b6bb-4efd-be8d-8fe867accf54 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:49.356833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912244f-1bfc-4963-b03b-ccdf39c74691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:49.357981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912244f-1bfc-4963-b03b-ccdf39c74691 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:49.358485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912244f-1bfc-4963-b03b-ccdf39c74691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:49.359001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912244f-1bfc-4963-b03b-ccdf39c74691 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:49.360447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912244f-1bfc-4963-b03b-ccdf39c74691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:49.363370 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912244f-1bfc-4963-b03b-ccdf39c74691 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:49.373032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:49.388304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 963/1925] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:49 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:49.393327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae644b-fa16-49b0-9a7c-e816e7d0db71 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:49.404336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae644b-fa16-49b0-9a7c-e816e7d0db71 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:49.409836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d9068-6523-43ef-bf66-5665bd0db9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:49.410508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d9068-6523-43ef-bf66-5665bd0db9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:49.410792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d9068-6523-43ef-bf66-5665bd0db9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:49.411072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d9068-6523-43ef-bf66-5665bd0db9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:49.411799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d9068-6523-43ef-bf66-5665bd0db9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:49.413460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d9068-6523-43ef-bf66-5665bd0db9f8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:49.424849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 963/1926] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:49.430988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912244f-1bfc-4963-b03b-ccdf39c74691 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:49.441946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912244f-1bfc-4963-b03b-ccdf39c74691 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:49.446924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4348c2-7d26-4c07-b55e-f39fec91074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:49.447612 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4348c2-7d26-4c07-b55e-f39fec91074b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:49.447894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4348c2-7d26-4c07-b55e-f39fec91074b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:49.448169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4348c2-7d26-4c07-b55e-f39fec91074b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:49.448902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4348c2-7d26-4c07-b55e-f39fec91074b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:49.450566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4348c2-7d26-4c07-b55e-f39fec91074b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:49.462764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 964/1927] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:49.469216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d9068-6523-43ef-bf66-5665bd0db9f8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:49.483903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d9068-6523-43ef-bf66-5665bd0db9f8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:49.488882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6423d3-2eb0-4005-af38-eeb680203f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:49.489850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6423d3-2eb0-4005-af38-eeb680203f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:49.490137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6423d3-2eb0-4005-af38-eeb680203f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:49.490419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6423d3-2eb0-4005-af38-eeb680203f10 admin admin] PATH_INFO: `/v3/projects/e0cd2cc8e04748bd8387b323e7ffeee7/users/1a5d556956fd470eb46a8503af6424fc/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:49.512734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6423d3-2eb0-4005-af38-eeb680203f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0cd2cc8e04748bd8387b323e7ffeee7, user_id=1a5d556956fd470eb46a8503af6424fc, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:49.516426 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6423d3-2eb0-4005-af38-eeb680203f10 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:49.534760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 964/1928] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:49 2022] PUT /identity/v3/projects/e0cd2cc8e04748bd8387b323e7ffeee7/users/1a5d556956fd470eb46a8503af6424fc/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:49.540614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2117c-ede6-434d-a5e1-fcca68a51bde None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:49.540948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2117c-ede6-434d-a5e1-fcca68a51bde None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:49.541321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2117c-ede6-434d-a5e1-fcca68a51bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:49.560181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d7b2117c-ede6-434d-a5e1-fcca68a51bde None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:49.561469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d7b2117c-ede6-434d-a5e1-fcca68a51bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:49.628330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 965/1929] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:49.638599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6423d3-2eb0-4005-af38-eeb680203f10 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:49.640521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cb6423d3-2eb0-4005-af38-eeb680203f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:49.711518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6423d3-2eb0-4005-af38-eeb680203f10 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:49.715559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76350a1a-bc2e-4b56-af08-74a52f79f730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:49.716159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76350a1a-bc2e-4b56-af08-74a52f79f730 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:49.716457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76350a1a-bc2e-4b56-af08-74a52f79f730 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:49.716742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76350a1a-bc2e-4b56-af08-74a52f79f730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:49.728136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76350a1a-bc2e-4b56-af08-74a52f79f730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:49.729692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76350a1a-bc2e-4b56-af08-74a52f79f730 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:49.740917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 965/1930] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:50.692165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fa116-255c-4524-bd8c-51e25c2a8a35 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:50.692977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fa116-255c-4524-bd8c-51e25c2a8a35 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:50.692977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fa116-255c-4524-bd8c-51e25c2a8a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:50.721572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e93fa116-255c-4524-bd8c-51e25c2a8a35 None None] MFA Rules not processed for user `1a5d556956fd470eb46a8503af6424fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:50.722798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e93fa116-255c-4524-bd8c-51e25c2a8a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:50.798742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 966/1931] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:50.803554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76350a1a-bc2e-4b56-af08-74a52f79f730 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:50.814745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76350a1a-bc2e-4b56-af08-74a52f79f730 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:50.821012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58712689-b3f8-42a0-a580-1b72619d6b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:50.821751 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58712689-b3f8-42a0-a580-1b72619d6b18 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:50.822046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58712689-b3f8-42a0-a580-1b72619d6b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:50.822330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58712689-b3f8-42a0-a580-1b72619d6b18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:50.823349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58712689-b3f8-42a0-a580-1b72619d6b18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:50.825405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58712689-b3f8-42a0-a580-1b72619d6b18 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:50.830139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:50.862527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 966/1932] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:50 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:50.866832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fa116-255c-4524-bd8c-51e25c2a8a35 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:50.878080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fa116-255c-4524-bd8c-51e25c2a8a35 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:50.939065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868d4b5-3b5e-4920-bff4-26ceedd77f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:50.939748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868d4b5-3b5e-4920-bff4-26ceedd77f39 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:50.940038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868d4b5-3b5e-4920-bff4-26ceedd77f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:50.940317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868d4b5-3b5e-4920-bff4-26ceedd77f39 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:50.941319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868d4b5-3b5e-4920-bff4-26ceedd77f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:50.943258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868d4b5-3b5e-4920-bff4-26ceedd77f39 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:50.964512 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 967/1933] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:50 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:50.969517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58712689-b3f8-42a0-a580-1b72619d6b18 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:50.980708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58712689-b3f8-42a0-a580-1b72619d6b18 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:50.985496 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1669b2-4451-4bc8-9a25-0f386195dd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:50.986179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1669b2-4451-4bc8-9a25-0f386195dd28 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:50.986466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1669b2-4451-4bc8-9a25-0f386195dd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:50.986753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1669b2-4451-4bc8-9a25-0f386195dd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:50.987494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1669b2-4451-4bc8-9a25-0f386195dd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:50.989128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1669b2-4451-4bc8-9a25-0f386195dd28 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:51.001035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 967/1934] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:51.005301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868d4b5-3b5e-4920-bff4-26ceedd77f39 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:51.010516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:51.017552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868d4b5-3b5e-4920-bff4-26ceedd77f39 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:51.024304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f483132-8af8-4fa4-a348-996ddc1adcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:51.025376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f483132-8af8-4fa4-a348-996ddc1adcae admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:51.025759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f483132-8af8-4fa4-a348-996ddc1adcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:51.026135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f483132-8af8-4fa4-a348-996ddc1adcae admin admin] PATH_INFO: `/v3/projects/c4f1e5dca8304135ade3e4c3baf2f975/users/79a91680bc3e45119ae283735a1e92b7/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:51.042792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f483132-8af8-4fa4-a348-996ddc1adcae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f1e5dca8304135ade3e4c3baf2f975, user_id=79a91680bc3e45119ae283735a1e92b7, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:51.045409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f483132-8af8-4fa4-a348-996ddc1adcae admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:51.059764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 968/1935] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:51 2022] PUT /identity/v3/projects/c4f1e5dca8304135ade3e4c3baf2f975/users/79a91680bc3e45119ae283735a1e92b7/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:51.064583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1669b2-4451-4bc8-9a25-0f386195dd28 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:51.077746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1669b2-4451-4bc8-9a25-0f386195dd28 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:51.142308 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fee1a-629a-496f-8530-b98f963137f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:51.142987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fee1a-629a-496f-8530-b98f963137f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:51.143273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fee1a-629a-496f-8530-b98f963137f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:51.143552 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fee1a-629a-496f-8530-b98f963137f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:51.144286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fee1a-629a-496f-8530-b98f963137f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:51.145955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fee1a-629a-496f-8530-b98f963137f4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:51.160681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 968/1936] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:51.162147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f483132-8af8-4fa4-a348-996ddc1adcae admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:51.179330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f483132-8af8-4fa4-a348-996ddc1adcae admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:51.188307 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c2533-fd54-4aa7-9276-3ade8451523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:51.189176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c2533-fd54-4aa7-9276-3ade8451523f admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:51.189476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c2533-fd54-4aa7-9276-3ade8451523f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:51.189760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c2533-fd54-4aa7-9276-3ade8451523f admin admin] PATH_INFO: `/v3/projects/c4f1e5dca8304135ade3e4c3baf2f975/users/79a91680bc3e45119ae283735a1e92b7/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:51.192161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c2533-fd54-4aa7-9276-3ade8451523f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f1e5dca8304135ade3e4c3baf2f975, user_id=79a91680bc3e45119ae283735a1e92b7, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:51.194872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c2533-fd54-4aa7-9276-3ade8451523f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:51.207943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 969/1937] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:51 2022] PUT /identity/v3/projects/c4f1e5dca8304135ade3e4c3baf2f975/users/79a91680bc3e45119ae283735a1e92b7/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:52.220846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9ff02-96ea-4a16-aa33-38d7705a80f3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:52.221718 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9ff02-96ea-4a16-aa33-38d7705a80f3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:52.221916 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9ff02-96ea-4a16-aa33-38d7705a80f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:52.290200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5ac9ff02-96ea-4a16-aa33-38d7705a80f3 None None] MFA Rules not processed for user `79a91680bc3e45119ae283735a1e92b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:52.292464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5ac9ff02-96ea-4a16-aa33-38d7705a80f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:52.495267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 969/1938] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:03:52.509324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c2533-fd54-4aa7-9276-3ade8451523f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:52.512556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ae8c2533-fd54-4aa7-9276-3ade8451523f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:52.618002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c2533-fd54-4aa7-9276-3ade8451523f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:52.625539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb75d9-cafa-4ec1-ac45-b343c9edfef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:52.627292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb75d9-cafa-4ec1-ac45-b343c9edfef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:52.627994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb75d9-cafa-4ec1-ac45-b343c9edfef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:52.628573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb75d9-cafa-4ec1-ac45-b343c9edfef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:52.744558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb75d9-cafa-4ec1-ac45-b343c9edfef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:52.762566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb75d9-cafa-4ec1-ac45-b343c9edfef7 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:52.800624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 970/1939] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:03:53.206026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9ff02-96ea-4a16-aa33-38d7705a80f3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:53.208515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5ac9ff02-96ea-4a16-aa33-38d7705a80f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:53.363293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:53.376317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9ff02-96ea-4a16-aa33-38d7705a80f3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:53.390584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:53.393959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:53.394978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:53.395648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] PATH_INFO: `/v3/users/1347dcdfd097453490b144bf22cbea76` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:53.399845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1347dcdfd097453490b144bf22cbea76)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:53.404206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:53.520987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1347dcdfd097453490b144bf22cbea76'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:53.541237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1347dcdfd097453490b144bf22cbea76'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:53.561396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1347dcdfd097453490b144bf22cbea76'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:53.578568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1347dcdfd097453490b144bf22cbea76'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:53.581316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1347dcdfd097453490b144bf22cbea76'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:53.591095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 970/1940] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:53 2022] DELETE /identity/v3/users/1347dcdfd097453490b144bf22cbea76 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:53.596750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb8abd-6e6e-4a27-9a99-1149fdcecdf7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:53.597259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb8abd-6e6e-4a27-9a99-1149fdcecdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:53.597595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb8abd-6e6e-4a27-9a99-1149fdcecdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:53.618879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-68fb8abd-6e6e-4a27-9a99-1149fdcecdf7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:53.620812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-68fb8abd-6e6e-4a27-9a99-1149fdcecdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:53.738633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 971/1941] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:53.756535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:53.758920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:53.846860 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c627dd-a5be-4007-a49a-0db3d3bf6eac admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:53.850848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb507c2-6624-400c-b8e2-10c4facacae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:53.851526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb507c2-6624-400c-b8e2-10c4facacae0 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:53.851814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb507c2-6624-400c-b8e2-10c4facacae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:53.852097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb507c2-6624-400c-b8e2-10c4facacae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:53.863471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb507c2-6624-400c-b8e2-10c4facacae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:53.865281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb507c2-6624-400c-b8e2-10c4facacae0 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:53.877206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 971/1942] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:54.204622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb8abd-6e6e-4a27-9a99-1149fdcecdf7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:54.206512 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-68fb8abd-6e6e-4a27-9a99-1149fdcecdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:54.221394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb8abd-6e6e-4a27-9a99-1149fdcecdf7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:54.228304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:54.229419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:54.229707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:54.229990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] PATH_INFO: `/v3/projects/bd0c3a0d5898414384ce780f51951d01` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:54.243046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0c3a0d5898414384ce780f51951d01)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:54.246415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:54.267416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:54.307166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0c3a0d5898414384ce780f51951d01', 'request_id': 'req-bcc508d8-aee8-4692-8183-8591147c6821'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.315143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0c3a0d5898414384ce780f51951d01', 'request_id': 'req-bcc508d8-aee8-4692-8183-8591147c6821'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.334106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0c3a0d5898414384ce780f51951d01', 'request_id': 'req-bcc508d8-aee8-4692-8183-8591147c6821'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.348280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] The token cache is being invalidate because project bd0c3a0d5898414384ce780f51951d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:54.348686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.349668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.351460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 972/1943] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:54 2022] DELETE /identity/v3/projects/bd0c3a0d5898414384ce780f51951d01 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:54.355615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb507c2-6624-400c-b8e2-10c4facacae0 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:54.369344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb507c2-6624-400c-b8e2-10c4facacae0 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:54.487750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:54.489779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:54.490421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:54.491715 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] PATH_INFO: `/v3/users/eda0bd8bde2a4dddbf8c23173490e2bb` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:54.495202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda0bd8bde2a4dddbf8c23173490e2bb)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:54.499349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:54.542797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:54.617480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda0bd8bde2a4dddbf8c23173490e2bb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.637457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda0bd8bde2a4dddbf8c23173490e2bb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.659442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda0bd8bde2a4dddbf8c23173490e2bb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.677048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda0bd8bde2a4dddbf8c23173490e2bb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.678256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-26bc9742-721e-4081-8d01-3d5ebc69018d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda0bd8bde2a4dddbf8c23173490e2bb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:54.693170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 972/1944] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:03:54 2022] DELETE /identity/v3/users/eda0bd8bde2a4dddbf8c23173490e2bb => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:55.111575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:55.112248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:55.135707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc508d8-aee8-4692-8183-8591147c6821 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:55.209514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:55.210396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.210801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.211087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] PATH_INFO: `/v3/projects/8cbe803de7d341fe9c62dbe9549c207e` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.213291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbe803de7d341fe9c62dbe9549c207e)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:55.215893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:55.278671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbe803de7d341fe9c62dbe9549c207e', 'request_id': 'req-d7609b83-98d5-4073-b9cc-4c7627fbfb77'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:55.288404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbe803de7d341fe9c62dbe9549c207e', 'request_id': 'req-d7609b83-98d5-4073-b9cc-4c7627fbfb77'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:55.299557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbe803de7d341fe9c62dbe9549c207e', 'request_id': 'req-d7609b83-98d5-4073-b9cc-4c7627fbfb77'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:55.312671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] The token cache is being invalidate because project 8cbe803de7d341fe9c62dbe9549c207e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:03:55.312965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:55.313689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7609b83-98d5-4073-b9cc-4c7627fbfb77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:03:55.315280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 973/1945] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:03:55 2022] DELETE /identity/v3/projects/8cbe803de7d341fe9c62dbe9549c207e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:03:55.322838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860c70c-4703-4d15-b24f-d5e1133cff77 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.323287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860c70c-4703-4d15-b24f-d5e1133cff77 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.323750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860c70c-4703-4d15-b24f-d5e1133cff77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.342969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6860c70c-4703-4d15-b24f-d5e1133cff77 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:55.344200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6860c70c-4703-4d15-b24f-d5e1133cff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:55.418651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 973/1946] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:55.426907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8a928-2c31-4e65-9c41-3ef21d640a01 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.427310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8a928-2c31-4e65-9c41-3ef21d640a01 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.427686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8a928-2c31-4e65-9c41-3ef21d640a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.447333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-91a8a928-2c31-4e65-9c41-3ef21d640a01 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:55.448553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-91a8a928-2c31-4e65-9c41-3ef21d640a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:55.457504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 974/1947] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:55.461994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860c70c-4703-4d15-b24f-d5e1133cff77 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:55.478747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860c70c-4703-4d15-b24f-d5e1133cff77 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:55.486202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65961355-208a-41b9-9e33-3a500c2b2e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:55.487270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65961355-208a-41b9-9e33-3a500c2b2e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.487748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65961355-208a-41b9-9e33-3a500c2b2e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.488213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65961355-208a-41b9-9e33-3a500c2b2e14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.489630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65961355-208a-41b9-9e33-3a500c2b2e14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:03:55.490023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65961355-208a-41b9-9e33-3a500c2b2e14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:55.492650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65961355-208a-41b9-9e33-3a500c2b2e14 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:55.513266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 974/1948] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:03:55.517837 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8a928-2c31-4e65-9c41-3ef21d640a01 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:55.541956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8a928-2c31-4e65-9c41-3ef21d640a01 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:55.549606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847dc0b-c365-4469-9ee2-b8d4adcf59cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:55.554216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847dc0b-c365-4469-9ee2-b8d4adcf59cf admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.554216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847dc0b-c365-4469-9ee2-b8d4adcf59cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.554216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847dc0b-c365-4469-9ee2-b8d4adcf59cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.554216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847dc0b-c365-4469-9ee2-b8d4adcf59cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:55.556048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847dc0b-c365-4469-9ee2-b8d4adcf59cf admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:55.599148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 975/1949] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:55 2022] POST /identity/v3/projects => generated 365 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:55.600765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65961355-208a-41b9-9e33-3a500c2b2e14 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:55.618763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65961355-208a-41b9-9e33-3a500c2b2e14 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:55.719377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef875c-3b80-456a-b49a-aa8dfaeab0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:55.720759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef875c-3b80-456a-b49a-aa8dfaeab0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.721048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef875c-3b80-456a-b49a-aa8dfaeab0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.721366 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef875c-3b80-456a-b49a-aa8dfaeab0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.722104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef875c-3b80-456a-b49a-aa8dfaeab0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:55.724103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef875c-3b80-456a-b49a-aa8dfaeab0d7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:55.749351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 975/1950] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:55 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:03:55.754443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847dc0b-c365-4469-9ee2-b8d4adcf59cf admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:55.761260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:55.765935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847dc0b-c365-4469-9ee2-b8d4adcf59cf admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:55.775573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323bf7f-ad19-422e-97e0-00e949c7af5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:55.777217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323bf7f-ad19-422e-97e0-00e949c7af5f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.778944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323bf7f-ad19-422e-97e0-00e949c7af5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.778944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323bf7f-ad19-422e-97e0-00e949c7af5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.787362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323bf7f-ad19-422e-97e0-00e949c7af5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:55.790357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323bf7f-ad19-422e-97e0-00e949c7af5f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:55.816151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 976/1951] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:03:55.818172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef875c-3b80-456a-b49a-aa8dfaeab0d7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:55.828852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef875c-3b80-456a-b49a-aa8dfaeab0d7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:55.834463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071998dd-e5e2-441d-b063-08e1b728c215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:55.835154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071998dd-e5e2-441d-b063-08e1b728c215 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.835441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071998dd-e5e2-441d-b063-08e1b728c215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.835741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071998dd-e5e2-441d-b063-08e1b728c215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.836474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071998dd-e5e2-441d-b063-08e1b728c215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:55.838156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071998dd-e5e2-441d-b063-08e1b728c215 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:55.850931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 976/1952] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:03:55.857764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323bf7f-ad19-422e-97e0-00e949c7af5f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:55.874750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323bf7f-ad19-422e-97e0-00e949c7af5f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:55.884340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f1d4c-78fc-41f4-91c0-efd696cece63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:55.885347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f1d4c-78fc-41f4-91c0-efd696cece63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.885706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f1d4c-78fc-41f4-91c0-efd696cece63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.886076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f1d4c-78fc-41f4-91c0-efd696cece63 admin admin] PATH_INFO: `/v3/projects/7735c439f9834715a51b9371fc014294/users/e40ad7459e924b0bb325e60afd0ef9b0/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.910186 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f1d4c-78fc-41f4-91c0-efd696cece63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7735c439f9834715a51b9371fc014294, user_id=e40ad7459e924b0bb325e60afd0ef9b0, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:55.912909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f1d4c-78fc-41f4-91c0-efd696cece63 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:55.927803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 977/1953] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:55 2022] PUT /identity/v3/projects/7735c439f9834715a51b9371fc014294/users/e40ad7459e924b0bb325e60afd0ef9b0/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:55.939463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7e639-9975-466a-a214-f3f57f1ed870 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:55.939758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7e639-9975-466a-a214-f3f57f1ed870 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:55.940040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7e639-9975-466a-a214-f3f57f1ed870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:55.948467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:55.976676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b2e7e639-9975-466a-a214-f3f57f1ed870 None None] MFA Rules not processed for user `e40ad7459e924b0bb325e60afd0ef9b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:55.977917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b2e7e639-9975-466a-a214-f3f57f1ed870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:56.061478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 977/1954] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:56.066627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f1d4c-78fc-41f4-91c0-efd696cece63 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:56.154509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f1d4c-78fc-41f4-91c0-efd696cece63 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:56.160268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab9128-7c84-48cd-9ea5-e9f62e48824a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:56.160958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab9128-7c84-48cd-9ea5-e9f62e48824a admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:56.161290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab9128-7c84-48cd-9ea5-e9f62e48824a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:56.161577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab9128-7c84-48cd-9ea5-e9f62e48824a admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:56.162330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab9128-7c84-48cd-9ea5-e9f62e48824a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:56.164294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab9128-7c84-48cd-9ea5-e9f62e48824a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:56.208378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 978/1955] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:03:56 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:56.209725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7e639-9975-466a-a214-f3f57f1ed870 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:56.222007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7e639-9975-466a-a214-f3f57f1ed870 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:56.299126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdeeed2-7fe2-4931-bc52-92e57f98553d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:56.300214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdeeed2-7fe2-4931-bc52-92e57f98553d admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:56.300730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdeeed2-7fe2-4931-bc52-92e57f98553d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:56.301434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdeeed2-7fe2-4931-bc52-92e57f98553d admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:56.302767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdeeed2-7fe2-4931-bc52-92e57f98553d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:56.306308 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdeeed2-7fe2-4931-bc52-92e57f98553d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:56.346172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 978/1956] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:03:56 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:03:56.350835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab9128-7c84-48cd-9ea5-e9f62e48824a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:56.370119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab9128-7c84-48cd-9ea5-e9f62e48824a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:56.378509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3721f-9f99-4140-8346-642f4d8a4809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:56.379662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3721f-9f99-4140-8346-642f4d8a4809 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:56.380173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3721f-9f99-4140-8346-642f4d8a4809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:56.380666 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3721f-9f99-4140-8346-642f4d8a4809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:56.382095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3721f-9f99-4140-8346-642f4d8a4809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:56.385739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3721f-9f99-4140-8346-642f4d8a4809 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:56.412621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 979/1957] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:56.414696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdeeed2-7fe2-4931-bc52-92e57f98553d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:56.433327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdeeed2-7fe2-4931-bc52-92e57f98553d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:56.441134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83a55f-0ebe-4bdf-a17e-b85f00bbe0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:56.442387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83a55f-0ebe-4bdf-a17e-b85f00bbe0fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:56.442813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83a55f-0ebe-4bdf-a17e-b85f00bbe0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:56.443101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83a55f-0ebe-4bdf-a17e-b85f00bbe0fc admin admin] PATH_INFO: `/v3/projects/275c06ce374e4005a5c2112f219ed00a/users/71bf7ca76bb84a1080e83b10eedda3ef/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:56.467488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83a55f-0ebe-4bdf-a17e-b85f00bbe0fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=275c06ce374e4005a5c2112f219ed00a, user_id=71bf7ca76bb84a1080e83b10eedda3ef, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:56.471763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83a55f-0ebe-4bdf-a17e-b85f00bbe0fc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:56.492166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 979/1958] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:56 2022] PUT /identity/v3/projects/275c06ce374e4005a5c2112f219ed00a/users/71bf7ca76bb84a1080e83b10eedda3ef/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:56.494794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3721f-9f99-4140-8346-642f4d8a4809 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:56.508959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3721f-9f99-4140-8346-642f4d8a4809 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:56.573294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124c2de-a671-4c28-bab6-fb3d54ab4b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:56.573884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124c2de-a671-4c28-bab6-fb3d54ab4b56 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:56.574173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124c2de-a671-4c28-bab6-fb3d54ab4b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:56.574452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124c2de-a671-4c28-bab6-fb3d54ab4b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:56.575181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124c2de-a671-4c28-bab6-fb3d54ab4b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:56.576896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124c2de-a671-4c28-bab6-fb3d54ab4b56 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:56.589942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 980/1959] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:03:56.594506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83a55f-0ebe-4bdf-a17e-b85f00bbe0fc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:56.605791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83a55f-0ebe-4bdf-a17e-b85f00bbe0fc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:56.610925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b983cc-55da-45d8-87c5-7c38e876a134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:56.612097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b983cc-55da-45d8-87c5-7c38e876a134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:56.612404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b983cc-55da-45d8-87c5-7c38e876a134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:56.612680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b983cc-55da-45d8-87c5-7c38e876a134 admin admin] PATH_INFO: `/v3/projects/275c06ce374e4005a5c2112f219ed00a/users/71bf7ca76bb84a1080e83b10eedda3ef/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:56.614833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b983cc-55da-45d8-87c5-7c38e876a134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275c06ce374e4005a5c2112f219ed00a, user_id=71bf7ca76bb84a1080e83b10eedda3ef, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:56.617574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b983cc-55da-45d8-87c5-7c38e876a134 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:56.630154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 980/1960] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:03:56 2022] PUT /identity/v3/projects/275c06ce374e4005a5c2112f219ed00a/users/71bf7ca76bb84a1080e83b10eedda3ef/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:03:56.640594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528169-9fac-4157-8edc-9d39a2d2c083 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:56.640885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528169-9fac-4157-8edc-9d39a2d2c083 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:56.641223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528169-9fac-4157-8edc-9d39a2d2c083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:56.675639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-91528169-9fac-4157-8edc-9d39a2d2c083 None None] MFA Rules not processed for user `71bf7ca76bb84a1080e83b10eedda3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:03:56.677319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-91528169-9fac-4157-8edc-9d39a2d2c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:56.732927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:56.801439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 981/1961] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:03:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:03:56.818412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b983cc-55da-45d8-87c5-7c38e876a134 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:56.820662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-90b983cc-55da-45d8-87c5-7c38e876a134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:56.933382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:03:56.947317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b983cc-55da-45d8-87c5-7c38e876a134 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:56.955150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b9d6-9bf9-459b-b105-081c0ff68491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:56.956432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b9d6-9bf9-459b-b105-081c0ff68491 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:56.956934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b9d6-9bf9-459b-b105-081c0ff68491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:56.958251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b9d6-9bf9-459b-b105-081c0ff68491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:56.969785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480b9d6-9bf9-459b-b105-081c0ff68491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:56.971360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480b9d6-9bf9-459b-b105-081c0ff68491 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:56.982546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 981/1962] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:03:57.092815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91528169-9fac-4157-8edc-9d39a2d2c083 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:57.104158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91528169-9fac-4157-8edc-9d39a2d2c083 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:57.108233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524808ee-fcf5-429b-8439-f601c0eb57fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:57.108233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524808ee-fcf5-429b-8439-f601c0eb57fe service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:57.109505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524808ee-fcf5-429b-8439-f601c0eb57fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:57.109505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524808ee-fcf5-429b-8439-f601c0eb57fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:57.176688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524808ee-fcf5-429b-8439-f601c0eb57fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:57.178190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524808ee-fcf5-429b-8439-f601c0eb57fe service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:57.188876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 982/1963] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:03:58.946240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b9d6-9bf9-459b-b105-081c0ff68491 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:03:58.948898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8480b9d6-9bf9-459b-b105-081c0ff68491 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:58.961828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b9d6-9bf9-459b-b105-081c0ff68491 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:03:58.966098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00b8ba-198f-4363-a6d5-5e36285b0456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:03:58.966846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00b8ba-198f-4363-a6d5-5e36285b0456 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:03:58.967201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00b8ba-198f-4363-a6d5-5e36285b0456 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:03:58.967548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00b8ba-198f-4363-a6d5-5e36285b0456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:03:58.969899 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-dd00b8ba-198f-4363-a6d5-5e36285b0456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:03:59.076095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00b8ba-198f-4363-a6d5-5e36285b0456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:03:59.079813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00b8ba-198f-4363-a6d5-5e36285b0456 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:03:59.094555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 982/1964] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:03:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:07.183643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=44 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:08.322334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524808ee-fcf5-429b-8439-f601c0eb57fe service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:08.441262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524808ee-fcf5-429b-8439-f601c0eb57fe service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:08.467001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:08.471859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:08.472362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:08.474902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] PATH_INFO: `/v3/users/e40ad7459e924b0bb325e60afd0ef9b0` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:08.480981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e40ad7459e924b0bb325e60afd0ef9b0)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:08.487809 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:08.631897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e40ad7459e924b0bb325e60afd0ef9b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:08.655606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e40ad7459e924b0bb325e60afd0ef9b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:08.675912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e40ad7459e924b0bb325e60afd0ef9b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:08.677177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e40ad7459e924b0bb325e60afd0ef9b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:08.694706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e40ad7459e924b0bb325e60afd0ef9b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:08.709822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 983/1965] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:08 2022] DELETE /identity/v3/users/e40ad7459e924b0bb325e60afd0ef9b0 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:08.715361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24216c-3a4c-4d82-b280-f3db7c9910da None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:08.716228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24216c-3a4c-4d82-b280-f3db7c9910da None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:08.716228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24216c-3a4c-4d82-b280-f3db7c9910da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:08.740752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6d24216c-3a4c-4d82-b280-f3db7c9910da None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:08.742040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6d24216c-3a4c-4d82-b280-f3db7c9910da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:08.826917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 983/1966] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:08.837814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:08.839573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:08.933361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6e6a0-670e-4a3d-aba7-70756fb871d9 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:08.947296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e25f0-d4f2-438a-8e11-9a8b41618be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:08.949538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e25f0-d4f2-438a-8e11-9a8b41618be9 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:08.950126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e25f0-d4f2-438a-8e11-9a8b41618be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:08.950637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e25f0-d4f2-438a-8e11-9a8b41618be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:08.971310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e25f0-d4f2-438a-8e11-9a8b41618be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:08.974174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e25f0-d4f2-438a-8e11-9a8b41618be9 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:08.992477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 984/1967] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:04:09.419141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24216c-3a4c-4d82-b280-f3db7c9910da None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:09.422278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6d24216c-3a4c-4d82-b280-f3db7c9910da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:09.452788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24216c-3a4c-4d82-b280-f3db7c9910da None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:09.463247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:09.465240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:09.466427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:09.467817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] PATH_INFO: `/v3/projects/7735c439f9834715a51b9371fc014294` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:09.473877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7735c439f9834715a51b9371fc014294)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:09.479224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:09.619176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7735c439f9834715a51b9371fc014294', 'request_id': 'req-65842a9e-e9c6-4633-bf0c-70b2312560b3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:09.635922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7735c439f9834715a51b9371fc014294', 'request_id': 'req-65842a9e-e9c6-4633-bf0c-70b2312560b3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:09.655147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7735c439f9834715a51b9371fc014294', 'request_id': 'req-65842a9e-e9c6-4633-bf0c-70b2312560b3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:09.705223 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] The token cache is being invalidate because project 7735c439f9834715a51b9371fc014294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:09.706492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:09.712470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:09.722482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 984/1968] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:09 2022] DELETE /identity/v3/projects/7735c439f9834715a51b9371fc014294 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:09.727723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e25f0-d4f2-438a-8e11-9a8b41618be9 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:09.759216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e25f0-d4f2-438a-8e11-9a8b41618be9 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:09.856462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:09.972188 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:09.973165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:09.973519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:09.973899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] PATH_INFO: `/v3/users/71bf7ca76bb84a1080e83b10eedda3ef` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:09.975483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bf7ca76bb84a1080e83b10eedda3ef)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:09.978472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:10.069866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bf7ca76bb84a1080e83b10eedda3ef'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.091304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bf7ca76bb84a1080e83b10eedda3ef'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.108357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bf7ca76bb84a1080e83b10eedda3ef'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.110559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bf7ca76bb84a1080e83b10eedda3ef'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.128690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f49cca0e-b6ed-4c52-b0e8-9e760caec40c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bf7ca76bb84a1080e83b10eedda3ef'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.136426 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 985/1969] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:09 2022] DELETE /identity/v3/users/71bf7ca76bb84a1080e83b10eedda3ef => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:04:10.633067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:10.634638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:10.658677 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65842a9e-e9c6-4633-bf0c-70b2312560b3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:10.676707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:10.768379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:10.769924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:10.770440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:10.770931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] PATH_INFO: `/v3/projects/275c06ce374e4005a5c2112f219ed00a` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:10.773357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=275c06ce374e4005a5c2112f219ed00a)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:10.777602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:10.905911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275c06ce374e4005a5c2112f219ed00a', 'request_id': 'req-f3130231-783a-43e1-8821-e4d551e57554'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.922981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275c06ce374e4005a5c2112f219ed00a', 'request_id': 'req-f3130231-783a-43e1-8821-e4d551e57554'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.941908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275c06ce374e4005a5c2112f219ed00a', 'request_id': 'req-f3130231-783a-43e1-8821-e4d551e57554'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.958749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] The token cache is being invalidate because project 275c06ce374e4005a5c2112f219ed00a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:10.959240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.960714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:10.963412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 985/1970] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:10 2022] DELETE /identity/v3/projects/275c06ce374e4005a5c2112f219ed00a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:10.968176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39d967-7f56-43f9-8962-770f995e49af None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:10.969537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39d967-7f56-43f9-8962-770f995e49af None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:10.969537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39d967-7f56-43f9-8962-770f995e49af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:10.990648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5a39d967-7f56-43f9-8962-770f995e49af None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:10.991909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5a39d967-7f56-43f9-8962-770f995e49af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:11.010569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:11.012111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:11.084631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 986/1971] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:11.086316 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816bb5-8ed4-474e-a96b-9d0300c889e8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.086614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816bb5-8ed4-474e-a96b-9d0300c889e8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.087093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816bb5-8ed4-474e-a96b-9d0300c889e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.121480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3130231-783a-43e1-8821-e4d551e57554 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:11.127841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:11.132835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.133446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.133839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] PATH_INFO: `/v3/users/bba06789d6a44f6ba31e4d368b55878b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.135404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bba06789d6a44f6ba31e4d368b55878b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:11.138105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:11.139464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-dd816bb5-8ed4-474e-a96b-9d0300c889e8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:11.143413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dd816bb5-8ed4-474e-a96b-9d0300c889e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:11.168842 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 987/1972] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:11.173363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd816bb5-8ed4-474e-a96b-9d0300c889e8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:11.187952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd816bb5-8ed4-474e-a96b-9d0300c889e8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:11.195948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8078f-bc17-4b1b-a715-867c79edfba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:11.197175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8078f-bc17-4b1b-a715-867c79edfba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.197665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8078f-bc17-4b1b-a715-867c79edfba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.198101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8078f-bc17-4b1b-a715-867c79edfba3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.199159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8078f-bc17-4b1b-a715-867c79edfba3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:04:11.199765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8078f-bc17-4b1b-a715-867c79edfba3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:11.203016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8078f-bc17-4b1b-a715-867c79edfba3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:11.234118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:11.243010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 988/1973] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:04:11.244923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8078f-bc17-4b1b-a715-867c79edfba3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:11.246595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bba06789d6a44f6ba31e4d368b55878b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:11.276937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8078f-bc17-4b1b-a715-867c79edfba3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:11.284539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bba06789d6a44f6ba31e4d368b55878b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:11.293806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1583393-6b85-4fd1-8701-c08f73d5260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:11.294528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1583393-6b85-4fd1-8701-c08f73d5260e admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.296711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1583393-6b85-4fd1-8701-c08f73d5260e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.296711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1583393-6b85-4fd1-8701-c08f73d5260e admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.297403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1583393-6b85-4fd1-8701-c08f73d5260e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:11.299991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1583393-6b85-4fd1-8701-c08f73d5260e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:11.310817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bba06789d6a44f6ba31e4d368b55878b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:11.363586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:11.366404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bba06789d6a44f6ba31e4d368b55878b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:11.368352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bba06789d6a44f6ba31e4d368b55878b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:11.389477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 986/1974] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:11 2022] DELETE /identity/v3/users/bba06789d6a44f6ba31e4d368b55878b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:11.390394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 989/1975] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:11 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:11.397806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da470d4-f760-4853-a62f-241f251aafbd None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.398110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da470d4-f760-4853-a62f-241f251aafbd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.398404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da470d4-f760-4853-a62f-241f251aafbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.399152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:11.402482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:11.433044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0da470d4-f760-4853-a62f-241f251aafbd None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:11.434328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0da470d4-f760-4853-a62f-241f251aafbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:11.622095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082bb7b-f20c-49e4-af1b-ed3f82269b36 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:11.626865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 990/1976] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:11.629181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da470d4-f760-4853-a62f-241f251aafbd None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:11.637803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f32c5-ca9f-49c8-b845-0857ec5cd4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:11.639569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f32c5-ca9f-49c8-b845-0857ec5cd4bc admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.640019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f32c5-ca9f-49c8-b845-0857ec5cd4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.640474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f32c5-ca9f-49c8-b845-0857ec5cd4bc admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.641955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f32c5-ca9f-49c8-b845-0857ec5cd4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:11.644070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f32c5-ca9f-49c8-b845-0857ec5cd4bc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:11.654192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0da470d4-f760-4853-a62f-241f251aafbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:11.707130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 987/1977] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:11 2022] POST /identity/v3/users => generated 288 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:11.713043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f32c5-ca9f-49c8-b845-0857ec5cd4bc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:11.727728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f32c5-ca9f-49c8-b845-0857ec5cd4bc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:11.734161 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd3389-4456-46df-8b75-35214070ba01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:11.734903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd3389-4456-46df-8b75-35214070ba01 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.735190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd3389-4456-46df-8b75-35214070ba01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.735470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd3389-4456-46df-8b75-35214070ba01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.736945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd3389-4456-46df-8b75-35214070ba01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:11.738686 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd3389-4456-46df-8b75-35214070ba01 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:11.764944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 988/1978] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:11.771940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd3389-4456-46df-8b75-35214070ba01 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:11.799295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd3389-4456-46df-8b75-35214070ba01 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:11.809519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498d6dc-b81f-466e-b4eb-4ef75c32587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:11.810541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498d6dc-b81f-466e-b4eb-4ef75c32587d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.810817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498d6dc-b81f-466e-b4eb-4ef75c32587d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.811359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498d6dc-b81f-466e-b4eb-4ef75c32587d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.812105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498d6dc-b81f-466e-b4eb-4ef75c32587d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:11.814801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498d6dc-b81f-466e-b4eb-4ef75c32587d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:11.830780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da470d4-f760-4853-a62f-241f251aafbd None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:11.833244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 989/1979] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:11.835516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498d6dc-b81f-466e-b4eb-4ef75c32587d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:11.838768 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c79eb4-0794-4148-8888-58dabe28df8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:11.839778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c79eb4-0794-4148-8888-58dabe28df8c service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.840266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c79eb4-0794-4148-8888-58dabe28df8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.840732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c79eb4-0794-4148-8888-58dabe28df8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.846287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498d6dc-b81f-466e-b4eb-4ef75c32587d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:11.853004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67fe6-7dec-43bd-bdcb-12aa2ee412d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:11.854287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c79eb4-0794-4148-8888-58dabe28df8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:11.854426 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67fe6-7dec-43bd-bdcb-12aa2ee412d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.854562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67fe6-7dec-43bd-bdcb-12aa2ee412d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.854781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67fe6-7dec-43bd-bdcb-12aa2ee412d1 admin admin] PATH_INFO: `/v3/projects/2bb5c29776e042f0b7469529d2b4ce34/users/ce129f79120642fc9614cbf43b38bc0f/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.855678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c79eb4-0794-4148-8888-58dabe28df8c service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:11.869613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 991/1980] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:11.902701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d67fe6-7dec-43bd-bdcb-12aa2ee412d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb5c29776e042f0b7469529d2b4ce34, user_id=ce129f79120642fc9614cbf43b38bc0f, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:11.908454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d67fe6-7dec-43bd-bdcb-12aa2ee412d1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:11.938810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 990/1981] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:11 2022] PUT /identity/v3/projects/2bb5c29776e042f0b7469529d2b4ce34/users/ce129f79120642fc9614cbf43b38bc0f/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:04:11.952290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e8c2f-9028-4bd2-99a6-ddaead0e4b1e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:11.952735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e8c2f-9028-4bd2-99a6-ddaead0e4b1e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:11.953364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e8c2f-9028-4bd2-99a6-ddaead0e4b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:11.999765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f71e8c2f-9028-4bd2-99a6-ddaead0e4b1e None None] MFA Rules not processed for user `ce129f79120642fc9614cbf43b38bc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:12.001935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f71e8c2f-9028-4bd2-99a6-ddaead0e4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:12.093148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 992/1982] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:12.114986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67fe6-7dec-43bd-bdcb-12aa2ee412d1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:12.118002 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-80d67fe6-7dec-43bd-bdcb-12aa2ee412d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:12.302897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67fe6-7dec-43bd-bdcb-12aa2ee412d1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:12.310095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b6b34-1458-43f5-a4d8-4a65d33a7bb0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915704f26c814eaa9e6c2a68533a9c66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:12.311577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b6b34-1458-43f5-a4d8-4a65d33a7bb0 service glance] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:12.312164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b6b34-1458-43f5-a4d8-4a65d33a7bb0 service glance] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:12.313477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b6b34-1458-43f5-a4d8-4a65d33a7bb0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:12.344087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b6b34-1458-43f5-a4d8-4a65d33a7bb0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:12.346029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b6b34-1458-43f5-a4d8-4a65d33a7bb0 service glance] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:12.368147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 991/1983] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:04:12.380912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e8c2f-9028-4bd2-99a6-ddaead0e4b1e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:12.382871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f71e8c2f-9028-4bd2-99a6-ddaead0e4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:12.480407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:12.569626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e8c2f-9028-4bd2-99a6-ddaead0e4b1e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:12.583034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:12.585125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:12.585125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:12.585125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] PATH_INFO: `/v3/projects/f0574a9b7bb24827a309629b0a24b1fd` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:12.591582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0574a9b7bb24827a309629b0a24b1fd)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:12.599641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:12.719883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0574a9b7bb24827a309629b0a24b1fd', 'request_id': 'req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:12.730568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0574a9b7bb24827a309629b0a24b1fd', 'request_id': 'req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:12.748131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0574a9b7bb24827a309629b0a24b1fd', 'request_id': 'req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:12.786598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] The token cache is being invalidate because project f0574a9b7bb24827a309629b0a24b1fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:12.787046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:12.788359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d5ff0372-fb86-4dd6-b5c1-08eb73d5147a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:12.794881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 993/1984] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:12 2022] DELETE /identity/v3/projects/f0574a9b7bb24827a309629b0a24b1fd => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:12.796979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed76242-5cd4-4b82-ac02-00653d422706 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:12.797809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed76242-5cd4-4b82-ac02-00653d422706 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:12.798194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed76242-5cd4-4b82-ac02-00653d422706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:12.815178 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:12.868597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-aed76242-5cd4-4b82-ac02-00653d422706 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:12.873134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-aed76242-5cd4-4b82-ac02-00653d422706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:13.049148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 992/1985] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:13.060099 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206fe0f-7c64-4f25-a63a-56fec2f20a6d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.061170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206fe0f-7c64-4f25-a63a-56fec2f20a6d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.061602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206fe0f-7c64-4f25-a63a-56fec2f20a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.102878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4206fe0f-7c64-4f25-a63a-56fec2f20a6d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:13.104584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4206fe0f-7c64-4f25-a63a-56fec2f20a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:13.114912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 994/1986] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:13.120441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed76242-5cd4-4b82-ac02-00653d422706 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:13.137279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed76242-5cd4-4b82-ac02-00653d422706 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:13.145676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3c99c-3de3-40c4-823e-c175d09abcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:13.146953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3c99c-3de3-40c4-823e-c175d09abcab admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.147450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3c99c-3de3-40c4-823e-c175d09abcab admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.147886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3c99c-3de3-40c4-823e-c175d09abcab admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.148729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3c99c-3de3-40c4-823e-c175d09abcab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:04:13.149141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3c99c-3de3-40c4-823e-c175d09abcab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:13.151000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3c99c-3de3-40c4-823e-c175d09abcab admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:13.169921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 993/1987] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:04:13.171097 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206fe0f-7c64-4f25-a63a-56fec2f20a6d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:13.185007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206fe0f-7c64-4f25-a63a-56fec2f20a6d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:13.194324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a501280-9ec4-4dd4-9d0c-6d45161ad7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:13.194324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a501280-9ec4-4dd4-9d0c-6d45161ad7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.194324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a501280-9ec4-4dd4-9d0c-6d45161ad7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.194690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a501280-9ec4-4dd4-9d0c-6d45161ad7d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.195275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a501280-9ec4-4dd4-9d0c-6d45161ad7d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:13.197532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a501280-9ec4-4dd4-9d0c-6d45161ad7d2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:13.241535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 995/1988] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:13 2022] POST /identity/v3/projects => generated 397 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:13.243340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3c99c-3de3-40c4-823e-c175d09abcab admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:13.257045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3c99c-3de3-40c4-823e-c175d09abcab admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:13.299224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a501280-9ec4-4dd4-9d0c-6d45161ad7d2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:13.301526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1a501280-9ec4-4dd4-9d0c-6d45161ad7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:13.416405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa0ec2-9050-42af-9c6f-b5cb803c5117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:13.419376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa0ec2-9050-42af-9c6f-b5cb803c5117 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.420103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa0ec2-9050-42af-9c6f-b5cb803c5117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.421041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa0ec2-9050-42af-9c6f-b5cb803c5117 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.424942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa0ec2-9050-42af-9c6f-b5cb803c5117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:13.431790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa0ec2-9050-42af-9c6f-b5cb803c5117 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:13.473452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a501280-9ec4-4dd4-9d0c-6d45161ad7d2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:13.483006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:13.485063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.485614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.486132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] PATH_INFO: `/v3/users/73a501cc0a6445709533da2b17de8a54` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.488682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a501cc0a6445709533da2b17de8a54)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:13.493366 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:13.500943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 994/1989] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:13 2022] POST /identity/v3/users => generated 299 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:13.503552 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa0ec2-9050-42af-9c6f-b5cb803c5117 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:13.522192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa0ec2-9050-42af-9c6f-b5cb803c5117 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:13.543944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53d60e-cbbe-4081-8743-97d25ea5a360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:13.545598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53d60e-cbbe-4081-8743-97d25ea5a360 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.546097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53d60e-cbbe-4081-8743-97d25ea5a360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.546586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53d60e-cbbe-4081-8743-97d25ea5a360 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.549070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53d60e-cbbe-4081-8743-97d25ea5a360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:13.552513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53d60e-cbbe-4081-8743-97d25ea5a360 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:13.579452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 995/1990] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:13.581056 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53d60e-cbbe-4081-8743-97d25ea5a360 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:13.584628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:13.599719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53d60e-cbbe-4081-8743-97d25ea5a360 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:13.607990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee88939-8715-49d0-abb4-6b775c9abb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:13.608986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee88939-8715-49d0-abb4-6b775c9abb7d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.609119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a501cc0a6445709533da2b17de8a54'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:13.609936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee88939-8715-49d0-abb4-6b775c9abb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.610268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee88939-8715-49d0-abb4-6b775c9abb7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.612277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee88939-8715-49d0-abb4-6b775c9abb7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:13.614799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee88939-8715-49d0-abb4-6b775c9abb7d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:13.637927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 996/1991] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:13.639226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee88939-8715-49d0-abb4-6b775c9abb7d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:13.650334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a501cc0a6445709533da2b17de8a54'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:13.650887 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee88939-8715-49d0-abb4-6b775c9abb7d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:13.656267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92673991-a9cf-4ddf-9055-bddbc263093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:13.657744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92673991-a9cf-4ddf-9055-bddbc263093f admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.657975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92673991-a9cf-4ddf-9055-bddbc263093f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.658409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92673991-a9cf-4ddf-9055-bddbc263093f admin admin] PATH_INFO: `/v3/projects/be3b9ef7cc9e4e57a123004258706689/users/fbab719c262747a4b97143b46d16daa5/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.666644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a501cc0a6445709533da2b17de8a54'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:13.667746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a501cc0a6445709533da2b17de8a54'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:13.682464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d7071141-4d90-4760-9b96-149ac9d165db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a501cc0a6445709533da2b17de8a54'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:13.697197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 996/1992] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:13 2022] DELETE /identity/v3/users/73a501cc0a6445709533da2b17de8a54 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:13.702537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4b401-42ac-4af4-bf27-64e21604495b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.702830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4b401-42ac-4af4-bf27-64e21604495b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.703663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4b401-42ac-4af4-bf27-64e21604495b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.714410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92673991-a9cf-4ddf-9055-bddbc263093f admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3b9ef7cc9e4e57a123004258706689, user_id=fbab719c262747a4b97143b46d16daa5, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:13.729558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92673991-a9cf-4ddf-9055-bddbc263093f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:13.763132 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 997/1993] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:13 2022] PUT /identity/v3/projects/be3b9ef7cc9e4e57a123004258706689/users/fbab719c262747a4b97143b46d16daa5/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:13.764349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f0f4b401-42ac-4af4-bf27-64e21604495b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:13.765727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f0f4b401-42ac-4af4-bf27-64e21604495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:13.774493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33459fe1-c4ea-4272-ae28-c3a96f95c9dc None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:13.774839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33459fe1-c4ea-4272-ae28-c3a96f95c9dc None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:13.775128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33459fe1-c4ea-4272-ae28-c3a96f95c9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:13.808078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-33459fe1-c4ea-4272-ae28-c3a96f95c9dc None None] MFA Rules not processed for user `fbab719c262747a4b97143b46d16daa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:13.810115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-33459fe1-c4ea-4272-ae28-c3a96f95c9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:13.851672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:13.944485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 997/1994] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:13.953965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4b401-42ac-4af4-bf27-64e21604495b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:13.956970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f0f4b401-42ac-4af4-bf27-64e21604495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:14.086123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 998/1995] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:14.099601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33459fe1-c4ea-4272-ae28-c3a96f95c9dc None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:14.102728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-33459fe1-c4ea-4272-ae28-c3a96f95c9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:14.157538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4b401-42ac-4af4-bf27-64e21604495b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:14.167322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d7986-5dab-473b-aeb8-8cca76b742f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:14.167322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d7986-5dab-473b-aeb8-8cca76b742f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:14.167322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d7986-5dab-473b-aeb8-8cca76b742f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:14.171386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d7986-5dab-473b-aeb8-8cca76b742f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:14.183211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d7986-5dab-473b-aeb8-8cca76b742f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:14.184672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d7986-5dab-473b-aeb8-8cca76b742f3 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:14.206165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 998/1996] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:04:14.223922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33459fe1-c4ea-4272-ae28-c3a96f95c9dc None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:14.227712 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e2b1e-88db-4cd6-8eeb-bbac18e4dcdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:14.228383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e2b1e-88db-4cd6-8eeb-bbac18e4dcdb service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:14.229346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e2b1e-88db-4cd6-8eeb-bbac18e4dcdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:14.229646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e2b1e-88db-4cd6-8eeb-bbac18e4dcdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:14.241556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e2b1e-88db-4cd6-8eeb-bbac18e4dcdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:14.243286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e2b1e-88db-4cd6-8eeb-bbac18e4dcdb service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:14.259119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 999/1997] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:15.010893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d7986-5dab-473b-aeb8-8cca76b742f3 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:15.012591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-816d7986-5dab-473b-aeb8-8cca76b742f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:15.036227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d7986-5dab-473b-aeb8-8cca76b742f3 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:15.060322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:15.061446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:15.061736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:15.062041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] PATH_INFO: `/v3/projects/e2484f487efa444b870f2d7d82cf9af0` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:15.064815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2484f487efa444b870f2d7d82cf9af0)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:15.067139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:15.178089 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2484f487efa444b870f2d7d82cf9af0', 'request_id': 'req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:15.182971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:15.196454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2484f487efa444b870f2d7d82cf9af0', 'request_id': 'req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:15.212622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2484f487efa444b870f2d7d82cf9af0', 'request_id': 'req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:15.237124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] The token cache is being invalidate because project e2484f487efa444b870f2d7d82cf9af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:15.239817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:15.251111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e8586c8-9bbc-4f35-a6ba-2f8eb21b2e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:15.258488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 999/1998] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:15 2022] DELETE /identity/v3/projects/e2484f487efa444b870f2d7d82cf9af0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:15.264353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f16888-900e-49fb-9c1e-01a928fddb0e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:15.265229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f16888-900e-49fb-9c1e-01a928fddb0e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:15.265229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f16888-900e-49fb-9c1e-01a928fddb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:15.295009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a9f16888-900e-49fb-9c1e-01a928fddb0e None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:15.297741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a9f16888-900e-49fb-9c1e-01a928fddb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:15.529519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1000/1999] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:15.539475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60627b2-67dc-4653-9a20-48d26a730ad4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:15.539909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60627b2-67dc-4653-9a20-48d26a730ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:15.540356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60627b2-67dc-4653-9a20-48d26a730ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:15.594463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f60627b2-67dc-4653-9a20-48d26a730ad4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:15.597638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f60627b2-67dc-4653-9a20-48d26a730ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:15.608145 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1000/2000] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:15.614629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f16888-900e-49fb-9c1e-01a928fddb0e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:15.638618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f16888-900e-49fb-9c1e-01a928fddb0e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:15.649491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc668592-86c1-4810-9f7c-154378a62e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:15.651956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc668592-86c1-4810-9f7c-154378a62e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:15.652473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc668592-86c1-4810-9f7c-154378a62e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:15.652981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc668592-86c1-4810-9f7c-154378a62e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:15.654459 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc668592-86c1-4810-9f7c-154378a62e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:04:15.655292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc668592-86c1-4810-9f7c-154378a62e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:15.660269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc668592-86c1-4810-9f7c-154378a62e37 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:15.689336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1001/2001] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:04:15.694020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60627b2-67dc-4653-9a20-48d26a730ad4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:15.707901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60627b2-67dc-4653-9a20-48d26a730ad4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:15.722705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dd681-6b0c-4406-a89a-9a08cd8a382f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:15.723864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dd681-6b0c-4406-a89a-9a08cd8a382f admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:15.724417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dd681-6b0c-4406-a89a-9a08cd8a382f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:15.725178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dd681-6b0c-4406-a89a-9a08cd8a382f admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:15.727903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dd681-6b0c-4406-a89a-9a08cd8a382f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:15.733961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dd681-6b0c-4406-a89a-9a08cd8a382f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:15.815108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1001/2002] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:15 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:15.821777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc668592-86c1-4810-9f7c-154378a62e37 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:15.845516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc668592-86c1-4810-9f7c-154378a62e37 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:15.972344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:15.990192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224f047-0541-4c24-b5e8-bd73131e236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:15.991675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224f047-0541-4c24-b5e8-bd73131e236a admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:15.992197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224f047-0541-4c24-b5e8-bd73131e236a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:15.992831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224f047-0541-4c24-b5e8-bd73131e236a admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:15.993844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224f047-0541-4c24-b5e8-bd73131e236a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:15.996479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224f047-0541-4c24-b5e8-bd73131e236a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:16.044205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1002/2003] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:15 2022] POST /identity/v3/users => generated 284 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:16.051059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dd681-6b0c-4406-a89a-9a08cd8a382f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:16.076909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dd681-6b0c-4406-a89a-9a08cd8a382f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:16.087374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de83d74-8882-4617-9098-c37d9dc17ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:16.090575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de83d74-8882-4617-9098-c37d9dc17ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:16.091317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de83d74-8882-4617-9098-c37d9dc17ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:16.092052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de83d74-8882-4617-9098-c37d9dc17ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:16.093782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de83d74-8882-4617-9098-c37d9dc17ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:16.097908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de83d74-8882-4617-9098-c37d9dc17ecd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:16.125526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1002/2004] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:16.131370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224f047-0541-4c24-b5e8-bd73131e236a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:16.165425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224f047-0541-4c24-b5e8-bd73131e236a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:16.175767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92ac52-4cd4-4d67-a98b-e4d29d432566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:16.177752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92ac52-4cd4-4d67-a98b-e4d29d432566 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:16.178490 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92ac52-4cd4-4d67-a98b-e4d29d432566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:16.179129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92ac52-4cd4-4d67-a98b-e4d29d432566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:16.181073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92ac52-4cd4-4d67-a98b-e4d29d432566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:16.184856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92ac52-4cd4-4d67-a98b-e4d29d432566 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:16.217733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1003/2005] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:16.223426 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de83d74-8882-4617-9098-c37d9dc17ecd admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:16.238095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de83d74-8882-4617-9098-c37d9dc17ecd admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:16.246543 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b440e-f3b6-455f-b222-c34b3cf74610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:16.248609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b440e-f3b6-455f-b222-c34b3cf74610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:16.249156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b440e-f3b6-455f-b222-c34b3cf74610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:16.249708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b440e-f3b6-455f-b222-c34b3cf74610 admin admin] PATH_INFO: `/v3/projects/e4d9e7ae30ef42e189a3cf86e676b50d/users/08be6b83eec04f058d4c37b8f55a9fce/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:16.323015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b440e-f3b6-455f-b222-c34b3cf74610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d9e7ae30ef42e189a3cf86e676b50d, user_id=08be6b83eec04f058d4c37b8f55a9fce, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:16.341113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b440e-f3b6-455f-b222-c34b3cf74610 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:16.375248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1003/2006] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:16 2022] PUT /identity/v3/projects/e4d9e7ae30ef42e189a3cf86e676b50d/users/08be6b83eec04f058d4c37b8f55a9fce/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:16.395306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66285c31-625f-4ad9-8c4f-5f40816b78c6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:16.396089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66285c31-625f-4ad9-8c4f-5f40816b78c6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:16.396089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66285c31-625f-4ad9-8c4f-5f40816b78c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:16.431095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-66285c31-625f-4ad9-8c4f-5f40816b78c6 None None] MFA Rules not processed for user `08be6b83eec04f058d4c37b8f55a9fce`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:16.432224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-66285c31-625f-4ad9-8c4f-5f40816b78c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:16.673222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1004/2007] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:16 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:16.696802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b440e-f3b6-455f-b222-c34b3cf74610 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:16.699982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-554b440e-f3b6-455f-b222-c34b3cf74610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:16.861746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b440e-f3b6-455f-b222-c34b3cf74610 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:16.868826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef39aa1-e285-4088-b1d7-f1532251eb65 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915704f26c814eaa9e6c2a68533a9c66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:16.869709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef39aa1-e285-4088-b1d7-f1532251eb65 service glance] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:16.870062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef39aa1-e285-4088-b1d7-f1532251eb65 service glance] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:16.870465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef39aa1-e285-4088-b1d7-f1532251eb65 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:16.884896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef39aa1-e285-4088-b1d7-f1532251eb65 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:16.887813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef39aa1-e285-4088-b1d7-f1532251eb65 service glance] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:16.925554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1004/2008] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:18.854580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933ef2e-dab2-419d-8c55-b4519651282f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:18.856669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933ef2e-dab2-419d-8c55-b4519651282f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:18.857170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933ef2e-dab2-419d-8c55-b4519651282f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:18.859557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1005/2009] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 19:04:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:04:18.868843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bcbab-208a-437f-a928-2c420fe8f53d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:18.869373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bcbab-208a-437f-a928-2c420fe8f53d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:18.869373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bcbab-208a-437f-a928-2c420fe8f53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:18.896369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-175bcbab-208a-437f-a928-2c420fe8f53d None None] MFA Rules not processed for user `f0e3fc619dc74a0591d8ad14369ae673`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:18.897842 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-175bcbab-208a-437f-a928-2c420fe8f53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:19.065435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1005/2010] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 19:04:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:19.087385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933ef2e-dab2-419d-8c55-b4519651282f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:19.089479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4933ef2e-dab2-419d-8c55-b4519651282f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:19.115779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933ef2e-dab2-419d-8c55-b4519651282f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:19.122541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfeed5-76ea-4932-9a9e-75e4e47a53bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:19.122818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfeed5-76ea-4932-9a9e-75e4e47a53bd service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:19.123123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfeed5-76ea-4932-9a9e-75e4e47a53bd service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:19.124590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfeed5-76ea-4932-9a9e-75e4e47a53bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:19.135873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfeed5-76ea-4932-9a9e-75e4e47a53bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:19.137573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfeed5-76ea-4932-9a9e-75e4e47a53bd service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:19.151788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1006/2011] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:19.685033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bcbab-208a-437f-a928-2c420fe8f53d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:19.687255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-175bcbab-208a-437f-a928-2c420fe8f53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:19.716573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:19.784247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bcbab-208a-437f-a928-2c420fe8f53d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:19.789447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:19.790469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:19.790894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:19.791173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] PATH_INFO: `/v3/users/1a5d556956fd470eb46a8503af6424fc` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:19.792499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a5d556956fd470eb46a8503af6424fc)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:19.794655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:19.847866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a5d556956fd470eb46a8503af6424fc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:19.860293 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a5d556956fd470eb46a8503af6424fc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:19.868231 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a5d556956fd470eb46a8503af6424fc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:19.868954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a5d556956fd470eb46a8503af6424fc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:19.877194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a5d556956fd470eb46a8503af6424fc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:19.883487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1006/2012] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:19 2022] DELETE /identity/v3/users/1a5d556956fd470eb46a8503af6424fc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:20.240707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfeed5-76ea-4932-9a9e-75e4e47a53bd service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:20.255101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfeed5-76ea-4932-9a9e-75e4e47a53bd service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:20.354815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:20.355600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:20.355906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:20.356169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] PATH_INFO: `/v3/projects/e0cd2cc8e04748bd8387b323e7ffeee7` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:20.357449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0cd2cc8e04748bd8387b323e7ffeee7)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:20.359983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:20.412059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0cd2cc8e04748bd8387b323e7ffeee7', 'request_id': 'req-538b73c1-6386-4d9f-819d-d288ac87c30c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.413248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:20.421444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0cd2cc8e04748bd8387b323e7ffeee7', 'request_id': 'req-538b73c1-6386-4d9f-819d-d288ac87c30c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.430755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0cd2cc8e04748bd8387b323e7ffeee7', 'request_id': 'req-538b73c1-6386-4d9f-819d-d288ac87c30c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.443106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] The token cache is being invalidate because project e0cd2cc8e04748bd8387b323e7ffeee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:20.443525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.444335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.446159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1007/2013] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:20 2022] DELETE /identity/v3/projects/e0cd2cc8e04748bd8387b323e7ffeee7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:04:20.450281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:20.451761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:20.465563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a197c22-0ce7-451b-a04e-3b555dd4fcc3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:20.529799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:20.530688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:20.531520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:20.531806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] PATH_INFO: `/v3/users/79a91680bc3e45119ae283735a1e92b7` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:20.533152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a91680bc3e45119ae283735a1e92b7)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:20.535154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:20.581843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:20.593379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a91680bc3e45119ae283735a1e92b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.611691 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a91680bc3e45119ae283735a1e92b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.624631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a91680bc3e45119ae283735a1e92b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.625331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a91680bc3e45119ae283735a1e92b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.633777 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bb51c2da-53ce-4125-81ed-ad36ee6a73e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a91680bc3e45119ae283735a1e92b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.641218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1007/2014] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:20 2022] DELETE /identity/v3/users/79a91680bc3e45119ae283735a1e92b7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:20.827364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:20.829568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:20.844277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b73c1-6386-4d9f-819d-d288ac87c30c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:20.912191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:20.913234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:20.913641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:20.914023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] PATH_INFO: `/v3/projects/c4f1e5dca8304135ade3e4c3baf2f975` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:20.915457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f1e5dca8304135ade3e4c3baf2f975)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:20.917640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:20.965996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f1e5dca8304135ade3e4c3baf2f975', 'request_id': 'req-2db5693d-f42f-44dd-a1d2-ee558630376e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.973080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f1e5dca8304135ade3e4c3baf2f975', 'request_id': 'req-2db5693d-f42f-44dd-a1d2-ee558630376e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.982179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f1e5dca8304135ade3e4c3baf2f975', 'request_id': 'req-2db5693d-f42f-44dd-a1d2-ee558630376e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.993279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] The token cache is being invalidate because project c4f1e5dca8304135ade3e4c3baf2f975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:20.993573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.994240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2db5693d-f42f-44dd-a1d2-ee558630376e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:20.995812 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1008/2015] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:20 2022] DELETE /identity/v3/projects/c4f1e5dca8304135ade3e4c3baf2f975 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:21.001724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d2023-56d8-49b6-8e42-a2c002dcc50e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.002234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d2023-56d8-49b6-8e42-a2c002dcc50e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.002330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d2023-56d8-49b6-8e42-a2c002dcc50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.021416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ad6d2023-56d8-49b6-8e42-a2c002dcc50e None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:21.022655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ad6d2023-56d8-49b6-8e42-a2c002dcc50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:21.102852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1008/2016] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:21.104950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfdc86-c92a-4edd-8448-985b2e0f452d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.105436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfdc86-c92a-4edd-8448-985b2e0f452d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.105673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfdc86-c92a-4edd-8448-985b2e0f452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.124509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d7bfdc86-c92a-4edd-8448-985b2e0f452d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:21.125743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d7bfdc86-c92a-4edd-8448-985b2e0f452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:21.134340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1009/2017] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:21.138805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d2023-56d8-49b6-8e42-a2c002dcc50e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:21.149236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d2023-56d8-49b6-8e42-a2c002dcc50e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:21.153863 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663d440-83c8-48df-acf4-205ec0bc651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:21.154521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663d440-83c8-48df-acf4-205ec0bc651d admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.154813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663d440-83c8-48df-acf4-205ec0bc651d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.155095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663d440-83c8-48df-acf4-205ec0bc651d admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.155821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663d440-83c8-48df-acf4-205ec0bc651d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:04:21.156723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663d440-83c8-48df-acf4-205ec0bc651d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:21.158374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663d440-83c8-48df-acf4-205ec0bc651d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:21.173142 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1009/2018] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:04:21.178452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfdc86-c92a-4edd-8448-985b2e0f452d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:21.189614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfdc86-c92a-4edd-8448-985b2e0f452d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:21.194745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc48e24-27f3-4d84-ac2c-8ca28de289c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:21.195416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc48e24-27f3-4d84-ac2c-8ca28de289c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.195705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc48e24-27f3-4d84-ac2c-8ca28de289c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.195986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc48e24-27f3-4d84-ac2c-8ca28de289c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.196717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc48e24-27f3-4d84-ac2c-8ca28de289c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:21.198756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc48e24-27f3-4d84-ac2c-8ca28de289c2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:21.230937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1010/2019] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:21 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:21.235718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663d440-83c8-48df-acf4-205ec0bc651d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:21.247692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663d440-83c8-48df-acf4-205ec0bc651d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:21.308590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b888f6-5d88-46d4-b139-d381d03dc2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:21.309300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b888f6-5d88-46d4-b139-d381d03dc2a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.309581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b888f6-5d88-46d4-b139-d381d03dc2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.309860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b888f6-5d88-46d4-b139-d381d03dc2a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.310562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b888f6-5d88-46d4-b139-d381d03dc2a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:21.313066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b888f6-5d88-46d4-b139-d381d03dc2a3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:21.342939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1010/2020] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:21 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:21.347477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc48e24-27f3-4d84-ac2c-8ca28de289c2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:21.359299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc48e24-27f3-4d84-ac2c-8ca28de289c2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:21.364521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d93012-2fb8-4a09-92fc-8f72215ed304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:21.365245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d93012-2fb8-4a09-92fc-8f72215ed304 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.365537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d93012-2fb8-4a09-92fc-8f72215ed304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.365818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d93012-2fb8-4a09-92fc-8f72215ed304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.366743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d93012-2fb8-4a09-92fc-8f72215ed304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:21.368504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d93012-2fb8-4a09-92fc-8f72215ed304 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:21.381144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1011/2021] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:21.387081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b888f6-5d88-46d4-b139-d381d03dc2a3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:21.402898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b888f6-5d88-46d4-b139-d381d03dc2a3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:21.409012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd95456-04c5-42a2-8932-69fa0c431716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:21.409733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd95456-04c5-42a2-8932-69fa0c431716 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.410026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd95456-04c5-42a2-8932-69fa0c431716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.410312 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd95456-04c5-42a2-8932-69fa0c431716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.411045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd95456-04c5-42a2-8932-69fa0c431716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:21.412721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd95456-04c5-42a2-8932-69fa0c431716 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:21.423931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1011/2022] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:21.430006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d93012-2fb8-4a09-92fc-8f72215ed304 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:21.443610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d93012-2fb8-4a09-92fc-8f72215ed304 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:21.448262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fcc31-30af-4e0b-ad3e-cff8d45fff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:21.449202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fcc31-30af-4e0b-ad3e-cff8d45fff88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.449491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fcc31-30af-4e0b-ad3e-cff8d45fff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.449773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fcc31-30af-4e0b-ad3e-cff8d45fff88 admin admin] PATH_INFO: `/v3/projects/997d35ce711b423f90b477fa518c4307/users/cd385a240b8a4727a8d5f13c48ffe412/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.467435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fcc31-30af-4e0b-ad3e-cff8d45fff88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=997d35ce711b423f90b477fa518c4307, user_id=cd385a240b8a4727a8d5f13c48ffe412, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:21.470218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fcc31-30af-4e0b-ad3e-cff8d45fff88 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:21.486636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1012/2023] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:21 2022] PUT /identity/v3/projects/997d35ce711b423f90b477fa518c4307/users/cd385a240b8a4727a8d5f13c48ffe412/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:21.506015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c688be-c33a-4cfd-8c56-dc642cabc9f3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.506310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c688be-c33a-4cfd-8c56-dc642cabc9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.506594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c688be-c33a-4cfd-8c56-dc642cabc9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.538868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e6c688be-c33a-4cfd-8c56-dc642cabc9f3 None None] MFA Rules not processed for user `cd385a240b8a4727a8d5f13c48ffe412`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:21.540105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e6c688be-c33a-4cfd-8c56-dc642cabc9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:21.549514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:21.617667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1012/2024] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:21.630045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fcc31-30af-4e0b-ad3e-cff8d45fff88 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:21.631906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f96fcc31-30af-4e0b-ad3e-cff8d45fff88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:21.638637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:21.704183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fcc31-30af-4e0b-ad3e-cff8d45fff88 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:21.708164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1135f58-d85b-414e-a7e1-7d84f3d41b51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:21.708828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1135f58-d85b-414e-a7e1-7d84f3d41b51 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:21.709205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1135f58-d85b-414e-a7e1-7d84f3d41b51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:21.709493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1135f58-d85b-414e-a7e1-7d84f3d41b51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:21.719241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1135f58-d85b-414e-a7e1-7d84f3d41b51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:21.720706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1135f58-d85b-414e-a7e1-7d84f3d41b51 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:21.730802 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1013/2025] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:26.464447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c688be-c33a-4cfd-8c56-dc642cabc9f3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:26.598015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c688be-c33a-4cfd-8c56-dc642cabc9f3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:26.610303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:26.612117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:26.612480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:26.612904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] PATH_INFO: `/v3/users/cd385a240b8a4727a8d5f13c48ffe412` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:26.614737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd385a240b8a4727a8d5f13c48ffe412)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:26.617933 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:26.688415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd385a240b8a4727a8d5f13c48ffe412'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.704312 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd385a240b8a4727a8d5f13c48ffe412'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.708613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1135f58-d85b-414e-a7e1-7d84f3d41b51 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:26.711329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a1135f58-d85b-414e-a7e1-7d84f3d41b51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:26.720409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd385a240b8a4727a8d5f13c48ffe412'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.721545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd385a240b8a4727a8d5f13c48ffe412'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.732931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1135f58-d85b-414e-a7e1-7d84f3d41b51 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:26.733303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e37d8a5e-2139-4498-9ed1-10b56bccf97a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd385a240b8a4727a8d5f13c48ffe412'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.742924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1013/2026] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:26 2022] DELETE /identity/v3/users/cd385a240b8a4727a8d5f13c48ffe412 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:26.743194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:26.744502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:26.744995 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:26.745547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] PATH_INFO: `/v3/users/fbab719c262747a4b97143b46d16daa5` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:26.748367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbab719c262747a4b97143b46d16daa5)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:26.749005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf45e7c-9c89-494c-954f-9fa16282c854 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:26.749616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf45e7c-9c89-494c-954f-9fa16282c854 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:26.750278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf45e7c-9c89-494c-954f-9fa16282c854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:26.751888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:26.771484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bbf45e7c-9c89-494c-954f-9fa16282c854 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:26.772747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bbf45e7c-9c89-494c-954f-9fa16282c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:26.809336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:26.825573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbab719c262747a4b97143b46d16daa5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.844087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbab719c262747a4b97143b46d16daa5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.854448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbab719c262747a4b97143b46d16daa5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.865250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1014/2027] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:26.867132 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbab719c262747a4b97143b46d16daa5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.868325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-449c5d3e-c105-43ba-b4ed-df05abf34a85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbab719c262747a4b97143b46d16daa5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:26.875668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf45e7c-9c89-494c-954f-9fa16282c854 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:26.877214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bbf45e7c-9c89-494c-954f-9fa16282c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:26.877486 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1014/2028] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:26 2022] DELETE /identity/v3/users/fbab719c262747a4b97143b46d16daa5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:26.881732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5603d9-1f53-4b3d-82ec-b6e33217801d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:26.882062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5603d9-1f53-4b3d-82ec-b6e33217801d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:26.882314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5603d9-1f53-4b3d-82ec-b6e33217801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:26.913126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7f5603d9-1f53-4b3d-82ec-b6e33217801d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:26.915261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7f5603d9-1f53-4b3d-82ec-b6e33217801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:26.939289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf45e7c-9c89-494c-954f-9fa16282c854 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:26.998545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1015/2029] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:27.009251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5603d9-1f53-4b3d-82ec-b6e33217801d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:27.010874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7f5603d9-1f53-4b3d-82ec-b6e33217801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:27.018449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:27.019469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:27.019469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:27.019702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:27.021673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:27.035950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:27.037875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:27.038998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:27.049305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1015/2030] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:27.095668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5603d9-1f53-4b3d-82ec-b6e33217801d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:27.099866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3bcc1-afcc-4442-b0c4-93aca98cbd40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:27.100540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3bcc1-afcc-4442-b0c4-93aca98cbd40 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:27.100824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3bcc1-afcc-4442-b0c4-93aca98cbd40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:27.101114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3bcc1-afcc-4442-b0c4-93aca98cbd40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:27.113368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3bcc1-afcc-4442-b0c4-93aca98cbd40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:27.116485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3bcc1-afcc-4442-b0c4-93aca98cbd40 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:27.137231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1016/2031] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:27.467635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:27.469201 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:27.495498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820e4d5-56ef-4e79-bf60-661313d700d3 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:27.504090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:27.505282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:27.505735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:27.506287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] PATH_INFO: `/v3/projects/997d35ce711b423f90b477fa518c4307` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:27.508058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=997d35ce711b423f90b477fa518c4307)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:27.510739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:27.654766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997d35ce711b423f90b477fa518c4307', 'request_id': 'req-156d8851-2883-42d7-a5d4-3e721689e859'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:27.673127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997d35ce711b423f90b477fa518c4307', 'request_id': 'req-156d8851-2883-42d7-a5d4-3e721689e859'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:27.684627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997d35ce711b423f90b477fa518c4307', 'request_id': 'req-156d8851-2883-42d7-a5d4-3e721689e859'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:27.706056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] The token cache is being invalidate because project 997d35ce711b423f90b477fa518c4307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:27.706320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:27.707191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:27.709072 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1016/2032] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:27 2022] DELETE /identity/v3/projects/997d35ce711b423f90b477fa518c4307 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:27.714660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63290f9d-fdd6-4524-a737-fee1f9fc5fd4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:27.715228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63290f9d-fdd6-4524-a737-fee1f9fc5fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:27.715228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63290f9d-fdd6-4524-a737-fee1f9fc5fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:27.735308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:27.736827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:27.755121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-63290f9d-fdd6-4524-a737-fee1f9fc5fd4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:27.756506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-63290f9d-fdd6-4524-a737-fee1f9fc5fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:27.770209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:27.856384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1017/2033] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:27.864695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d8851-2883-42d7-a5d4-3e721689e859 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:27.864987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a1539-9ec5-4c58-ab9d-83ee48c4ec53 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:27.865279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a1539-9ec5-4c58-ab9d-83ee48c4ec53 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:27.865524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a1539-9ec5-4c58-ab9d-83ee48c4ec53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:27.876222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:27.877159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:27.877442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:27.877723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] PATH_INFO: `/v3/projects/be3b9ef7cc9e4e57a123004258706689` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:27.879194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3b9ef7cc9e4e57a123004258706689)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:27.881575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:27.892235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-247a1539-9ec5-4c58-ab9d-83ee48c4ec53 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:27.893502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-247a1539-9ec5-4c58-ab9d-83ee48c4ec53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:27.901609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1018/2034] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:27.906882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a1539-9ec5-4c58-ab9d-83ee48c4ec53 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:27.919614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a1539-9ec5-4c58-ab9d-83ee48c4ec53 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:27.925965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed3f0d-89e4-4788-bbcc-c8c2db579a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:27.926777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed3f0d-89e4-4788-bbcc-c8c2db579a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:27.927250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed3f0d-89e4-4788-bbcc-c8c2db579a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:27.927613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed3f0d-89e4-4788-bbcc-c8c2db579a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:27.928402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed3f0d-89e4-4788-bbcc-c8c2db579a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:04:27.928719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed3f0d-89e4-4788-bbcc-c8c2db579a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:27.930441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed3f0d-89e4-4788-bbcc-c8c2db579a34 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:27.944779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1019/2035] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:04:27.949780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed3f0d-89e4-4788-bbcc-c8c2db579a34 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:27.957193 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3b9ef7cc9e4e57a123004258706689', 'request_id': 'req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:27.964073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed3f0d-89e4-4788-bbcc-c8c2db579a34 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:27.965384 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3b9ef7cc9e4e57a123004258706689', 'request_id': 'req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:27.976572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3b9ef7cc9e4e57a123004258706689', 'request_id': 'req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:28.002686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] The token cache is being invalidate because project be3b9ef7cc9e4e57a123004258706689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:28.003355 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:28.012623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8d0b9aa9-c73b-40cb-be0e-f7dee4924c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:28.015227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1017/2036] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:27 2022] DELETE /identity/v3/projects/be3b9ef7cc9e4e57a123004258706689 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:04:28.034570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203fd5e-0075-44ef-bc42-b3eac73ad449 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.035043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203fd5e-0075-44ef-bc42-b3eac73ad449 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.035538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203fd5e-0075-44ef-bc42-b3eac73ad449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.050747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018967d-b66c-4b37-b896-9441dafeb8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.051423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018967d-b66c-4b37-b896-9441dafeb8dd admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.051901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018967d-b66c-4b37-b896-9441dafeb8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.052432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018967d-b66c-4b37-b896-9441dafeb8dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.053284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018967d-b66c-4b37-b896-9441dafeb8dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.055233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018967d-b66c-4b37-b896-9441dafeb8dd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.060484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8203fd5e-0075-44ef-bc42-b3eac73ad449 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:28.062172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8203fd5e-0075-44ef-bc42-b3eac73ad449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:28.077255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1018/2037] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:28.081351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385dab1-6db6-4cbd-a1af-0fee3b059a7b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.081646 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385dab1-6db6-4cbd-a1af-0fee3b059a7b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.081926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385dab1-6db6-4cbd-a1af-0fee3b059a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.109803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1385dab1-6db6-4cbd-a1af-0fee3b059a7b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:28.111940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1020/2038] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:27 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:28.114055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1385dab1-6db6-4cbd-a1af-0fee3b059a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:28.117821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018967d-b66c-4b37-b896-9441dafeb8dd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.121481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1018967d-b66c-4b37-b896-9441dafeb8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:28.154849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:28.209276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1019/2039] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:28.210807 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385dab1-6db6-4cbd-a1af-0fee3b059a7b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.216765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018967d-b66c-4b37-b896-9441dafeb8dd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.223247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385dab1-6db6-4cbd-a1af-0fee3b059a7b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.224173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950528c7-0d0a-4386-a7d2-1ae2c696690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.225260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950528c7-0d0a-4386-a7d2-1ae2c696690f admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.225709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950528c7-0d0a-4386-a7d2-1ae2c696690f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.226163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950528c7-0d0a-4386-a7d2-1ae2c696690f admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.227053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950528c7-0d0a-4386-a7d2-1ae2c696690f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.229268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950528c7-0d0a-4386-a7d2-1ae2c696690f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.229432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f9b8a-4b35-4b06-a138-aa93202a953f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.230136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f9b8a-4b35-4b06-a138-aa93202a953f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.230425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f9b8a-4b35-4b06-a138-aa93202a953f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.230701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f9b8a-4b35-4b06-a138-aa93202a953f admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.231450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f9b8a-4b35-4b06-a138-aa93202a953f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:04:28.231761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f9b8a-4b35-4b06-a138-aa93202a953f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.233455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f9b8a-4b35-4b06-a138-aa93202a953f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.251739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1020/2040] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:04:28.256958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1021/2041] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:28 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:28.258112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f9b8a-4b35-4b06-a138-aa93202a953f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.262465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950528c7-0d0a-4386-a7d2-1ae2c696690f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.268590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f9b8a-4b35-4b06-a138-aa93202a953f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.274181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c91e-6d66-43d9-b8ce-be387364c408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.274860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c91e-6d66-43d9-b8ce-be387364c408 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.275143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c91e-6d66-43d9-b8ce-be387364c408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.275599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c91e-6d66-43d9-b8ce-be387364c408 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.276318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45c91e-6d66-43d9-b8ce-be387364c408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.278316 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45c91e-6d66-43d9-b8ce-be387364c408 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.281388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950528c7-0d0a-4386-a7d2-1ae2c696690f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.289355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636bd58-8a50-4db1-b1e4-3ecfba5a061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.290516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636bd58-8a50-4db1-b1e4-3ecfba5a061d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.291009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636bd58-8a50-4db1-b1e4-3ecfba5a061d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.291798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636bd58-8a50-4db1-b1e4-3ecfba5a061d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.293258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636bd58-8a50-4db1-b1e4-3ecfba5a061d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.296913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636bd58-8a50-4db1-b1e4-3ecfba5a061d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.304524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:28.318638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1021/2042] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:28 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:28.323355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1022/2043] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:28.324397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c91e-6d66-43d9-b8ce-be387364c408 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.326962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636bd58-8a50-4db1-b1e4-3ecfba5a061d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.339760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636bd58-8a50-4db1-b1e4-3ecfba5a061d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.408496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c91e-6d66-43d9-b8ce-be387364c408 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.413609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e64512-c672-4a2a-9cb1-5ad72826c846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.414308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e64512-c672-4a2a-9cb1-5ad72826c846 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.414621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e64512-c672-4a2a-9cb1-5ad72826c846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.414905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e64512-c672-4a2a-9cb1-5ad72826c846 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.415621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e64512-c672-4a2a-9cb1-5ad72826c846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.417617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e64512-c672-4a2a-9cb1-5ad72826c846 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.422134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c30280-e7f2-4462-9d71-1483540fae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.423145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c30280-e7f2-4462-9d71-1483540fae09 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.423576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c30280-e7f2-4462-9d71-1483540fae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.424003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c30280-e7f2-4462-9d71-1483540fae09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.425206 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c30280-e7f2-4462-9d71-1483540fae09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.427892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c30280-e7f2-4462-9d71-1483540fae09 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.441155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1022/2044] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:28 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:28.447359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1023/2045] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:28.447623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e64512-c672-4a2a-9cb1-5ad72826c846 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.448566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c30280-e7f2-4462-9d71-1483540fae09 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.462923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e64512-c672-4a2a-9cb1-5ad72826c846 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.463128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c30280-e7f2-4462-9d71-1483540fae09 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.469325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7efe3-0009-41d5-be03-49a6797593df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.469561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a3379-9c28-4b15-978d-723a60a37fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.470255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a3379-9c28-4b15-978d-723a60a37fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.470401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7efe3-0009-41d5-be03-49a6797593df admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.470549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a3379-9c28-4b15-978d-723a60a37fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.470709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7efe3-0009-41d5-be03-49a6797593df admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.470823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a3379-9c28-4b15-978d-723a60a37fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.470917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7efe3-0009-41d5-be03-49a6797593df admin admin] PATH_INFO: `/v3/projects/2ecae699aa154bf18e0b8638a7772ba9/users/41899c4d45394437aea347a4e4b56ea3/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.471755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a3379-9c28-4b15-978d-723a60a37fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.473494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a3379-9c28-4b15-978d-723a60a37fa6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.492966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1023/2046] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:28.499413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7efe3-0009-41d5-be03-49a6797593df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ecae699aa154bf18e0b8638a7772ba9, user_id=41899c4d45394437aea347a4e4b56ea3, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.499558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a3379-9c28-4b15-978d-723a60a37fa6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.502642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7efe3-0009-41d5-be03-49a6797593df admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.516819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a3379-9c28-4b15-978d-723a60a37fa6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.521551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1024/2047] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:28 2022] PUT /identity/v3/projects/2ecae699aa154bf18e0b8638a7772ba9/users/41899c4d45394437aea347a4e4b56ea3/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:28.550546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea64ee-2739-4aea-ace6-8a6b1f43246b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.550999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea64ee-2739-4aea-ace6-8a6b1f43246b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.551457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea64ee-2739-4aea-ace6-8a6b1f43246b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.589944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d9ea64ee-2739-4aea-ace6-8a6b1f43246b None None] MFA Rules not processed for user `41899c4d45394437aea347a4e4b56ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:28.591534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d9ea64ee-2739-4aea-ace6-8a6b1f43246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:28.602034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900509d7-db8c-4c13-954f-3a15dd4be83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.602428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900509d7-db8c-4c13-954f-3a15dd4be83d admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.602705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900509d7-db8c-4c13-954f-3a15dd4be83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.602990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900509d7-db8c-4c13-954f-3a15dd4be83d admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.603733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900509d7-db8c-4c13-954f-3a15dd4be83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.605679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900509d7-db8c-4c13-954f-3a15dd4be83d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.618261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1024/2048] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:28.622545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900509d7-db8c-4c13-954f-3a15dd4be83d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.634584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900509d7-db8c-4c13-954f-3a15dd4be83d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.640452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5757cb-e549-43fc-81d9-6abeda893782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.641403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5757cb-e549-43fc-81d9-6abeda893782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.641696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5757cb-e549-43fc-81d9-6abeda893782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.641979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5757cb-e549-43fc-81d9-6abeda893782 admin admin] PATH_INFO: `/v3/projects/cccd9a91f96a4a698f5d9d7028216a35/users/569256e6d1d1418eafb5ff2ed8909926/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.665605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5757cb-e549-43fc-81d9-6abeda893782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cccd9a91f96a4a698f5d9d7028216a35, user_id=569256e6d1d1418eafb5ff2ed8909926, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.668322 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5757cb-e549-43fc-81d9-6abeda893782 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.693194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1025/2049] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:28 2022] PUT /identity/v3/projects/cccd9a91f96a4a698f5d9d7028216a35/users/569256e6d1d1418eafb5ff2ed8909926/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:04:28.696252 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0736b99-e97a-4521-8f5a-44553af8990d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.696552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0736b99-e97a-4521-8f5a-44553af8990d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.696833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0736b99-e97a-4521-8f5a-44553af8990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.697356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1025/2050] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:28.719891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a0736b99-e97a-4521-8f5a-44553af8990d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:28.721049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a0736b99-e97a-4521-8f5a-44553af8990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:28.721884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea64ee-2739-4aea-ace6-8a6b1f43246b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.726663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d9ea64ee-2739-4aea-ace6-8a6b1f43246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:28.740265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:28.779359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:28.806604 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1026/2051] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:28.807742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea64ee-2739-4aea-ace6-8a6b1f43246b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.814439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0736b99-e97a-4521-8f5a-44553af8990d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:28.816714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a0736b99-e97a-4521-8f5a-44553af8990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:28.895052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bed45e-75e3-44d6-9522-1ea98a824194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.895729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bed45e-75e3-44d6-9522-1ea98a824194 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.896021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bed45e-75e3-44d6-9522-1ea98a824194 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.896305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bed45e-75e3-44d6-9522-1ea98a824194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.929931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0736b99-e97a-4521-8f5a-44553af8990d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:28.937811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a15a01-d942-4572-8872-cacf5f637a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:28.938314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a15a01-d942-4572-8872-cacf5f637a82 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:28.938791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a15a01-d942-4572-8872-cacf5f637a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:28.939251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a15a01-d942-4572-8872-cacf5f637a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:28.966060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a15a01-d942-4572-8872-cacf5f637a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:28.969324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a15a01-d942-4572-8872-cacf5f637a82 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:28.992439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1027/2052] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:29.015832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bed45e-75e3-44d6-9522-1ea98a824194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:29.018008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bed45e-75e3-44d6-9522-1ea98a824194 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:29.029330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1026/2053] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:29.062819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a15a01-d942-4572-8872-cacf5f637a82 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:29.079212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a15a01-d942-4572-8872-cacf5f637a82 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:29.087087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:29.087982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:29.088267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:29.088545 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] PATH_INFO: `/v3/users/41899c4d45394437aea347a4e4b56ea3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:29.090010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=41899c4d45394437aea347a4e4b56ea3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:29.091990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:29.165049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41899c4d45394437aea347a4e4b56ea3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:29.194770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41899c4d45394437aea347a4e4b56ea3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:29.209170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41899c4d45394437aea347a4e4b56ea3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:29.210311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41899c4d45394437aea347a4e4b56ea3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:29.226527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41899c4d45394437aea347a4e4b56ea3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:29.237923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1028/2054] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:29 2022] DELETE /identity/v3/users/41899c4d45394437aea347a4e4b56ea3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:29.244927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34940899-33be-456e-9543-7b300a60869a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:29.245484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34940899-33be-456e-9543-7b300a60869a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:29.245988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34940899-33be-456e-9543-7b300a60869a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:29.276952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-34940899-33be-456e-9543-7b300a60869a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:29.279362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-34940899-33be-456e-9543-7b300a60869a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:29.394348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1027/2055] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:29.411432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:29.414305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:29.495300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291048be-0361-4204-b52c-8fbb91dfd8be admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:29.500525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9541a1-ca89-416d-ab9f-83fe240f2a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:29.501329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9541a1-ca89-416d-ab9f-83fe240f2a14 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:29.501620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9541a1-ca89-416d-ab9f-83fe240f2a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:29.501901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9541a1-ca89-416d-ab9f-83fe240f2a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:29.512907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9541a1-ca89-416d-ab9f-83fe240f2a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:29.514437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9541a1-ca89-416d-ab9f-83fe240f2a14 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:29.528012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1029/2056] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:30.077074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34940899-33be-456e-9543-7b300a60869a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:30.079191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-34940899-33be-456e-9543-7b300a60869a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:30.129805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34940899-33be-456e-9543-7b300a60869a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:30.136103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:30.137129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:30.137425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:30.138003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] PATH_INFO: `/v3/projects/2ecae699aa154bf18e0b8638a7772ba9` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:30.139543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ecae699aa154bf18e0b8638a7772ba9)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:30.142288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:30.168735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:30.266866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ecae699aa154bf18e0b8638a7772ba9', 'request_id': 'req-30e869f3-2114-4927-a9a5-f04114ab0efe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:30.276602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ecae699aa154bf18e0b8638a7772ba9', 'request_id': 'req-30e869f3-2114-4927-a9a5-f04114ab0efe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:30.287201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ecae699aa154bf18e0b8638a7772ba9', 'request_id': 'req-30e869f3-2114-4927-a9a5-f04114ab0efe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:30.301059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] The token cache is being invalidate because project 2ecae699aa154bf18e0b8638a7772ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:30.301542 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:30.302330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:30.304210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1028/2057] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:30 2022] DELETE /identity/v3/projects/2ecae699aa154bf18e0b8638a7772ba9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:35.442404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c4480-8f22-46d9-b891-dee58dd9da6e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:35.443008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c4480-8f22-46d9-b891-dee58dd9da6e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:35.443205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c4480-8f22-46d9-b891-dee58dd9da6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:35.465903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:35.482312 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d55c4480-8f22-46d9-b891-dee58dd9da6e None None] MFA Rules not processed for user `569256e6d1d1418eafb5ff2ed8909926`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:35.485005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d55c4480-8f22-46d9-b891-dee58dd9da6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:35.591653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1030/2058] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:35.596083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:35.597667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:35.681453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e869f3-2114-4927-a9a5-f04114ab0efe admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:35.687546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba43e5-eaa8-43c9-b899-187374b13b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:35.688455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba43e5-eaa8-43c9-b899-187374b13b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:35.688842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba43e5-eaa8-43c9-b899-187374b13b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:35.689360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba43e5-eaa8-43c9-b899-187374b13b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:35.690335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba43e5-eaa8-43c9-b899-187374b13b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:35.693005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba43e5-eaa8-43c9-b899-187374b13b54 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:35.737965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1029/2059] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:35 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:35.742657 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c4480-8f22-46d9-b891-dee58dd9da6e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:35.761030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c4480-8f22-46d9-b891-dee58dd9da6e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:35.837323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c133f-b1ee-4356-87fd-a817b52935f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:35.838170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c133f-b1ee-4356-87fd-a817b52935f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:35.838460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c133f-b1ee-4356-87fd-a817b52935f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:35.838749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c133f-b1ee-4356-87fd-a817b52935f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:35.840108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c133f-b1ee-4356-87fd-a817b52935f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:35.843057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c133f-b1ee-4356-87fd-a817b52935f2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:35.871021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1031/2060] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:35 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:35.872352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba43e5-eaa8-43c9-b899-187374b13b54 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:35.887774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba43e5-eaa8-43c9-b899-187374b13b54 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:35.908046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d8591-bbe4-44de-955f-c82a4fda8577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:35.909211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d8591-bbe4-44de-955f-c82a4fda8577 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:35.909622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d8591-bbe4-44de-955f-c82a4fda8577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:35.910007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d8591-bbe4-44de-955f-c82a4fda8577 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:35.910928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d8591-bbe4-44de-955f-c82a4fda8577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:35.912808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d8591-bbe4-44de-955f-c82a4fda8577 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:35.932102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1030/2061] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:35.933492 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c133f-b1ee-4356-87fd-a817b52935f2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:35.946282 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c133f-b1ee-4356-87fd-a817b52935f2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:35.956162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e3bcc-90a8-47ec-9527-8212b439d77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:35.957074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e3bcc-90a8-47ec-9527-8212b439d77f admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:35.957394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e3bcc-90a8-47ec-9527-8212b439d77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:35.957905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e3bcc-90a8-47ec-9527-8212b439d77f admin admin] PATH_INFO: `/v3/projects/1f9ddf92bc3d46a18803d3493ce69eb9/users/d1a8d31ce48f4fb7bd22f4fe98f03565/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:35.979101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e3bcc-90a8-47ec-9527-8212b439d77f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f9ddf92bc3d46a18803d3493ce69eb9, user_id=d1a8d31ce48f4fb7bd22f4fe98f03565, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:35.982254 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e3bcc-90a8-47ec-9527-8212b439d77f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:36.002316 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1032/2062] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:35 2022] PUT /identity/v3/projects/1f9ddf92bc3d46a18803d3493ce69eb9/users/d1a8d31ce48f4fb7bd22f4fe98f03565/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:04:36.006278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d8591-bbe4-44de-955f-c82a4fda8577 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:36.017543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d8591-bbe4-44de-955f-c82a4fda8577 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:36.099152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c192199-112a-4da8-9e32-dfd5385d9d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:36.099896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c192199-112a-4da8-9e32-dfd5385d9d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:36.100393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c192199-112a-4da8-9e32-dfd5385d9d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:36.103008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c192199-112a-4da8-9e32-dfd5385d9d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:36.103008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c192199-112a-4da8-9e32-dfd5385d9d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:36.103572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c192199-112a-4da8-9e32-dfd5385d9d70 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:36.119185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1031/2063] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:36.120541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e3bcc-90a8-47ec-9527-8212b439d77f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:36.139405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e3bcc-90a8-47ec-9527-8212b439d77f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:36.144696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca75df-4cf3-4c98-8d99-019ad0945587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:36.145751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca75df-4cf3-4c98-8d99-019ad0945587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:36.146068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca75df-4cf3-4c98-8d99-019ad0945587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:36.146547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca75df-4cf3-4c98-8d99-019ad0945587 admin admin] PATH_INFO: `/v3/projects/1f9ddf92bc3d46a18803d3493ce69eb9/users/d1a8d31ce48f4fb7bd22f4fe98f03565/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:36.149065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca75df-4cf3-4c98-8d99-019ad0945587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f9ddf92bc3d46a18803d3493ce69eb9, user_id=d1a8d31ce48f4fb7bd22f4fe98f03565, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:36.152964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca75df-4cf3-4c98-8d99-019ad0945587 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:36.167553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1033/2064] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:36 2022] PUT /identity/v3/projects/1f9ddf92bc3d46a18803d3493ce69eb9/users/d1a8d31ce48f4fb7bd22f4fe98f03565/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:41.361031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c192199-112a-4da8-9e32-dfd5385d9d70 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:41.364337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2c192199-112a-4da8-9e32-dfd5385d9d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:41.395745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c192199-112a-4da8-9e32-dfd5385d9d70 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:41.411690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:41.510469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:41.512670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:41.513145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:41.513589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] PATH_INFO: `/v3/users/a7ee05285e124183a43f5757fd94ec40` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:41.515078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ee05285e124183a43f5757fd94ec40)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:41.518519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:41.619460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ee05285e124183a43f5757fd94ec40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:41.641042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ee05285e124183a43f5757fd94ec40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:41.652992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ee05285e124183a43f5757fd94ec40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:41.668649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ee05285e124183a43f5757fd94ec40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:41.669399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43010c28-46e8-47df-b1de-0125a0975336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ee05285e124183a43f5757fd94ec40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:41.679682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1032/2065] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:41 2022] DELETE /identity/v3/users/a7ee05285e124183a43f5757fd94ec40 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:41.960845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca75df-4cf3-4c98-8d99-019ad0945587 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:41.964118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a9ca75df-4cf3-4c98-8d99-019ad0945587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:41.995019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca75df-4cf3-4c98-8d99-019ad0945587 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:42.076684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:42.091564 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:42.093329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.093966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.094671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] PATH_INFO: `/v3/projects/d22f07bc3f5641d19b56a56257d0c951` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:42.096517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22f07bc3f5641d19b56a56257d0c951)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:42.098880 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:42.219891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22f07bc3f5641d19b56a56257d0c951', 'request_id': 'req-5e649095-6f54-45e3-948b-9ed349d3b32e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:42.234369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22f07bc3f5641d19b56a56257d0c951', 'request_id': 'req-5e649095-6f54-45e3-948b-9ed349d3b32e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:42.253876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22f07bc3f5641d19b56a56257d0c951', 'request_id': 'req-5e649095-6f54-45e3-948b-9ed349d3b32e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:42.286825 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] The token cache is being invalidate because project d22f07bc3f5641d19b56a56257d0c951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:42.287258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:42.288425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5e649095-6f54-45e3-948b-9ed349d3b32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:42.290379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1034/2066] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:41 2022] DELETE /identity/v3/projects/d22f07bc3f5641d19b56a56257d0c951 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:42.294979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d732fe-eef9-48e0-98f2-ca85fa494f20 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.295271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d732fe-eef9-48e0-98f2-ca85fa494f20 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.295547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d732fe-eef9-48e0-98f2-ca85fa494f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:42.345827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-50d732fe-eef9-48e0-98f2-ca85fa494f20 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:42.347236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-50d732fe-eef9-48e0-98f2-ca85fa494f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:42.454004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1033/2067] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:42.461900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bc315-4d23-4400-8173-26066ddef188 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.462191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bc315-4d23-4400-8173-26066ddef188 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.462481 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bc315-4d23-4400-8173-26066ddef188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:42.492517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-406bc315-4d23-4400-8173-26066ddef188 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:42.494593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-406bc315-4d23-4400-8173-26066ddef188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:42.507436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1035/2068] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:42.511844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d732fe-eef9-48e0-98f2-ca85fa494f20 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:42.525133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d732fe-eef9-48e0-98f2-ca85fa494f20 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:42.532245 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda67d1-96e3-4fd9-b932-868145a71294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:42.533016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda67d1-96e3-4fd9-b932-868145a71294 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.533532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda67d1-96e3-4fd9-b932-868145a71294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.534059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda67d1-96e3-4fd9-b932-868145a71294 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:42.535189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda67d1-96e3-4fd9-b932-868145a71294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:04:42.536080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda67d1-96e3-4fd9-b932-868145a71294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:42.538029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda67d1-96e3-4fd9-b932-868145a71294 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:42.551543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1034/2069] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:04:42.556011 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bc315-4d23-4400-8173-26066ddef188 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:42.579181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bc315-4d23-4400-8173-26066ddef188 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:42.587033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3094c-d0bd-44df-a885-c90110d5d5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:42.588153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3094c-d0bd-44df-a885-c90110d5d5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.588598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3094c-d0bd-44df-a885-c90110d5d5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.588882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3094c-d0bd-44df-a885-c90110d5d5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:42.589675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3094c-d0bd-44df-a885-c90110d5d5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:42.591828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3094c-d0bd-44df-a885-c90110d5d5d1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:42.642062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda67d1-96e3-4fd9-b932-868145a71294 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:42.642339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1036/2070] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:42 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:42.652866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda67d1-96e3-4fd9-b932-868145a71294 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:42.676921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:42.744454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4806e20-ae75-40f2-94bd-681dc5df8cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:42.745371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4806e20-ae75-40f2-94bd-681dc5df8cdb admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.745656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4806e20-ae75-40f2-94bd-681dc5df8cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.745922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4806e20-ae75-40f2-94bd-681dc5df8cdb admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:42.757720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4806e20-ae75-40f2-94bd-681dc5df8cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:42.761152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4806e20-ae75-40f2-94bd-681dc5df8cdb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:42.793331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1035/2071] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:42 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:42.795947 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3094c-d0bd-44df-a885-c90110d5d5d1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:42.820288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3094c-d0bd-44df-a885-c90110d5d5d1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:42.829118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee34744-9873-4b6a-8054-9a08a3ab1fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:42.830380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee34744-9873-4b6a-8054-9a08a3ab1fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.830842 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee34744-9873-4b6a-8054-9a08a3ab1fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.831245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee34744-9873-4b6a-8054-9a08a3ab1fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:42.832142 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee34744-9873-4b6a-8054-9a08a3ab1fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:42.834021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee34744-9873-4b6a-8054-9a08a3ab1fd8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:42.849255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1037/2072] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:42.853240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4806e20-ae75-40f2-94bd-681dc5df8cdb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:42.868825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4806e20-ae75-40f2-94bd-681dc5df8cdb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:42.874795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b6309-5a9b-461f-81c4-003f79e02a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:42.875770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b6309-5a9b-461f-81c4-003f79e02a3e admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.876234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b6309-5a9b-461f-81c4-003f79e02a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.876689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b6309-5a9b-461f-81c4-003f79e02a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:42.878265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b6309-5a9b-461f-81c4-003f79e02a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:42.880544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b6309-5a9b-461f-81c4-003f79e02a3e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:42.896661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1036/2073] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:42.901445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee34744-9873-4b6a-8054-9a08a3ab1fd8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:42.924756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee34744-9873-4b6a-8054-9a08a3ab1fd8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:42.929972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b762-420a-414a-8988-656fa1c3980f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:42.931177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b762-420a-414a-8988-656fa1c3980f admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.931617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b762-420a-414a-8988-656fa1c3980f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.932046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b762-420a-414a-8988-656fa1c3980f admin admin] PATH_INFO: `/v3/projects/9f2acd203aab474083cd9ac3271a24c0/users/5c0b35e6bb954d7b849e1251a4a130e0/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:42.953768 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519b762-420a-414a-8988-656fa1c3980f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2acd203aab474083cd9ac3271a24c0, user_id=5c0b35e6bb954d7b849e1251a4a130e0, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:42.956732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519b762-420a-414a-8988-656fa1c3980f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:42.976220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1038/2074] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:42 2022] PUT /identity/v3/projects/9f2acd203aab474083cd9ac3271a24c0/users/5c0b35e6bb954d7b849e1251a4a130e0/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:42.988278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea5820-89c7-4bbd-9bc5-e52183ab252f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:42.989199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea5820-89c7-4bbd-9bc5-e52183ab252f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:42.989419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea5820-89c7-4bbd-9bc5-e52183ab252f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:43.031973 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d5ea5820-89c7-4bbd-9bc5-e52183ab252f None None] MFA Rules not processed for user `5c0b35e6bb954d7b849e1251a4a130e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:43.033478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d5ea5820-89c7-4bbd-9bc5-e52183ab252f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:43.128334 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1037/2075] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:43.141740 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b762-420a-414a-8988-656fa1c3980f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:43.144616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7519b762-420a-414a-8988-656fa1c3980f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:43.227659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b762-420a-414a-8988-656fa1c3980f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:43.234501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d3ad2-dc5b-4ac1-8298-026345a57fec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:43.235354 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d3ad2-dc5b-4ac1-8298-026345a57fec service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:43.235842 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d3ad2-dc5b-4ac1-8298-026345a57fec service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:43.236287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d3ad2-dc5b-4ac1-8298-026345a57fec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:43.245920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d3ad2-dc5b-4ac1-8298-026345a57fec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:43.248284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d3ad2-dc5b-4ac1-8298-026345a57fec service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:43.270194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1039/2076] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:43.356873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0303b8-c811-40a9-a6bb-a02154ae551a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:43.357275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0303b8-c811-40a9-a6bb-a02154ae551a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:43.357906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0303b8-c811-40a9-a6bb-a02154ae551a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:43.393910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6c0303b8-c811-40a9-a6bb-a02154ae551a None None] MFA Rules not processed for user `d1a8d31ce48f4fb7bd22f4fe98f03565`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:43.395159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6c0303b8-c811-40a9-a6bb-a02154ae551a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:43.486098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1038/2077] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:43.650502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d3ad2-dc5b-4ac1-8298-026345a57fec service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:43.652031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-550d3ad2-dc5b-4ac1-8298-026345a57fec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:43.664893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:43.679401 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d3ad2-dc5b-4ac1-8298-026345a57fec service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:43.691066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20befb-aaae-45ba-8923-3896a61d8355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:43.694039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20befb-aaae-45ba-8923-3896a61d8355 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:43.694352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20befb-aaae-45ba-8923-3896a61d8355 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:43.694641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20befb-aaae-45ba-8923-3896a61d8355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:43.708977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9a20befb-aaae-45ba-8923-3896a61d8355 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:43.811905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20befb-aaae-45ba-8923-3896a61d8355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:43.813689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20befb-aaae-45ba-8923-3896a61d8355 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:43.826297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1040/2078] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:47.234379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0303b8-c811-40a9-a6bb-a02154ae551a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:47.248608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:47.314236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0303b8-c811-40a9-a6bb-a02154ae551a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:47.320069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:47.321019 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:47.321324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:47.321613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] PATH_INFO: `/v3/users/5c0b35e6bb954d7b849e1251a4a130e0` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:47.323380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c0b35e6bb954d7b849e1251a4a130e0)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:47.325952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:47.405505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c0b35e6bb954d7b849e1251a4a130e0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:47.425711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c0b35e6bb954d7b849e1251a4a130e0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:47.439329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c0b35e6bb954d7b849e1251a4a130e0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:47.452771 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c0b35e6bb954d7b849e1251a4a130e0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:47.454106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c0b35e6bb954d7b849e1251a4a130e0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:47.461696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1039/2079] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:47 2022] DELETE /identity/v3/users/5c0b35e6bb954d7b849e1251a4a130e0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:47.466418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ffe6-c6e7-40fd-9baa-ae540d48d6bf None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:47.466946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ffe6-c6e7-40fd-9baa-ae540d48d6bf None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:47.467046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ffe6-c6e7-40fd-9baa-ae540d48d6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:47.487339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9be2ffe6-c6e7-40fd-9baa-ae540d48d6bf None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:47.488816 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9be2ffe6-c6e7-40fd-9baa-ae540d48d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:47.560311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1041/2080] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:47.578220 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:47.579767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:47.653484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c66eca-1947-43bf-80d1-26f1200ad356 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:47.657679 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6e05e-27a8-4b0a-b405-9709fe56a457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:47.658335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6e05e-27a8-4b0a-b405-9709fe56a457 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:47.658620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6e05e-27a8-4b0a-b405-9709fe56a457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:47.658897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6e05e-27a8-4b0a-b405-9709fe56a457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:47.668627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6e05e-27a8-4b0a-b405-9709fe56a457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:47.670109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6e05e-27a8-4b0a-b405-9709fe56a457 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:47.680895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1040/2081] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:48.017366 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2ffe6-c6e7-40fd-9baa-ae540d48d6bf None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:48.036186 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9be2ffe6-c6e7-40fd-9baa-ae540d48d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:48.057131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2ffe6-c6e7-40fd-9baa-ae540d48d6bf None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:48.063363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:48.064260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.064555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.064898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] PATH_INFO: `/v3/projects/9f2acd203aab474083cd9ac3271a24c0` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.066497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f2acd203aab474083cd9ac3271a24c0)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:48.068439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:48.167379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f2acd203aab474083cd9ac3271a24c0', 'request_id': 'req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:48.175559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f2acd203aab474083cd9ac3271a24c0', 'request_id': 'req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:48.189655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f2acd203aab474083cd9ac3271a24c0', 'request_id': 'req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:48.207167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] The token cache is being invalidate because project 9f2acd203aab474083cd9ac3271a24c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:48.207741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:48.208590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4c1830b4-bdd4-443e-a266-5a9bbebc4cb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:48.214592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1042/2082] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:48 2022] DELETE /identity/v3/projects/9f2acd203aab474083cd9ac3271a24c0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:48.215513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a925800-889d-444e-932d-a51f81c68bfb None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.215813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a925800-889d-444e-932d-a51f81c68bfb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.216097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a925800-889d-444e-932d-a51f81c68bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.258780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1a925800-889d-444e-932d-a51f81c68bfb None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:48.262329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1a925800-889d-444e-932d-a51f81c68bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:48.338582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:48.379138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1041/2083] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:48.397559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae8f67-fbcb-487d-92f1-056ee541eec6 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.397934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae8f67-fbcb-487d-92f1-056ee541eec6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.398747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae8f67-fbcb-487d-92f1-056ee541eec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.422015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6bae8f67-fbcb-487d-92f1-056ee541eec6 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:48.423278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6bae8f67-fbcb-487d-92f1-056ee541eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:48.441202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1043/2084] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:48.444601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a925800-889d-444e-932d-a51f81c68bfb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:48.460438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a925800-889d-444e-932d-a51f81c68bfb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:48.466630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02111d-b79d-40ce-b998-b45c964e7c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:48.467360 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02111d-b79d-40ce-b998-b45c964e7c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.467652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02111d-b79d-40ce-b998-b45c964e7c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.467935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02111d-b79d-40ce-b998-b45c964e7c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.469468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02111d-b79d-40ce-b998-b45c964e7c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:04:48.469805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02111d-b79d-40ce-b998-b45c964e7c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:48.471473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02111d-b79d-40ce-b998-b45c964e7c96 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:48.485612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1042/2085] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:04:48.489970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae8f67-fbcb-487d-92f1-056ee541eec6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:48.501559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae8f67-fbcb-487d-92f1-056ee541eec6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:48.507261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607180d-63b7-4cb7-855f-a7875212d24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:48.507935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607180d-63b7-4cb7-855f-a7875212d24b admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.508220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607180d-63b7-4cb7-855f-a7875212d24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.508502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607180d-63b7-4cb7-855f-a7875212d24b admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.509246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607180d-63b7-4cb7-855f-a7875212d24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:48.511267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607180d-63b7-4cb7-855f-a7875212d24b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:48.530835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:48.552617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1044/2086] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:48 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:48.557071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02111d-b79d-40ce-b998-b45c964e7c96 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:48.569470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02111d-b79d-40ce-b998-b45c964e7c96 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:48.644485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184abd3-6b8a-4180-a8b6-bbf41f01274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:48.645359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184abd3-6b8a-4180-a8b6-bbf41f01274d admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.645753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184abd3-6b8a-4180-a8b6-bbf41f01274d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.646278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184abd3-6b8a-4180-a8b6-bbf41f01274d admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.647117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184abd3-6b8a-4180-a8b6-bbf41f01274d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:48.650080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184abd3-6b8a-4180-a8b6-bbf41f01274d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:48.677130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1043/2087] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:48 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:48.680306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607180d-63b7-4cb7-855f-a7875212d24b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:48.692386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607180d-63b7-4cb7-855f-a7875212d24b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:48.697356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94720065-c287-4d6b-b507-0d630b67132d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:48.698272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720065-c287-4d6b-b507-0d630b67132d admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.698591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720065-c287-4d6b-b507-0d630b67132d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.698879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720065-c287-4d6b-b507-0d630b67132d admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.699616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94720065-c287-4d6b-b507-0d630b67132d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:48.701281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94720065-c287-4d6b-b507-0d630b67132d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:48.719067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1045/2088] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:48.723143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184abd3-6b8a-4180-a8b6-bbf41f01274d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:48.734867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184abd3-6b8a-4180-a8b6-bbf41f01274d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:48.739747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c55bdd-24f8-4efd-b577-e1fe5d163ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:48.740531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c55bdd-24f8-4efd-b577-e1fe5d163ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.740820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c55bdd-24f8-4efd-b577-e1fe5d163ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.741139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c55bdd-24f8-4efd-b577-e1fe5d163ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.741878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c55bdd-24f8-4efd-b577-e1fe5d163ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:48.743493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c55bdd-24f8-4efd-b577-e1fe5d163ae5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:48.755761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1044/2089] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:48.759996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94720065-c287-4d6b-b507-0d630b67132d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:48.775856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94720065-c287-4d6b-b507-0d630b67132d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:48.785451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e718a6-2234-4eb5-abb6-0e9dd21c03c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:48.787181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e718a6-2234-4eb5-abb6-0e9dd21c03c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.787838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e718a6-2234-4eb5-abb6-0e9dd21c03c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.788382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e718a6-2234-4eb5-abb6-0e9dd21c03c8 admin admin] PATH_INFO: `/v3/projects/c320fb4b0cb0466594a57f59a295d481/users/4fd6674f93324d779a42763e361ad124/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.825181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e718a6-2234-4eb5-abb6-0e9dd21c03c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c320fb4b0cb0466594a57f59a295d481, user_id=4fd6674f93324d779a42763e361ad124, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:48.831501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e718a6-2234-4eb5-abb6-0e9dd21c03c8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:48.850720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1046/2090] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:48 2022] PUT /identity/v3/projects/c320fb4b0cb0466594a57f59a295d481/users/4fd6674f93324d779a42763e361ad124/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:48.862349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16de4-ea37-4772-82c2-e345c5cc53a1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:48.862563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16de4-ea37-4772-82c2-e345c5cc53a1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:48.862815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16de4-ea37-4772-82c2-e345c5cc53a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:48.901532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-05e16de4-ea37-4772-82c2-e345c5cc53a1 None None] MFA Rules not processed for user `4fd6674f93324d779a42763e361ad124`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:48.902943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-05e16de4-ea37-4772-82c2-e345c5cc53a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:49.003261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1045/2091] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:49.007675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e718a6-2234-4eb5-abb6-0e9dd21c03c8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.089066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e718a6-2234-4eb5-abb6-0e9dd21c03c8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.094531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716ea08-f605-4ff9-8567-1233be5a7bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.095201 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716ea08-f605-4ff9-8567-1233be5a7bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.095492 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716ea08-f605-4ff9-8567-1233be5a7bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.095776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716ea08-f605-4ff9-8567-1233be5a7bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.096482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716ea08-f605-4ff9-8567-1233be5a7bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.098435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716ea08-f605-4ff9-8567-1233be5a7bd7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.139029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1047/2092] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:49 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:49.140200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16de4-ea37-4772-82c2-e345c5cc53a1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.154267 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16de4-ea37-4772-82c2-e345c5cc53a1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.186669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:49.235310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941d29b-1381-4515-9ae9-dcee73ae6ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.236017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941d29b-1381-4515-9ae9-dcee73ae6ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.236416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941d29b-1381-4515-9ae9-dcee73ae6ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.236699 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941d29b-1381-4515-9ae9-dcee73ae6ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.237436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941d29b-1381-4515-9ae9-dcee73ae6ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.239416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941d29b-1381-4515-9ae9-dcee73ae6ca6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.264865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1046/2093] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:49 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:49.269196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716ea08-f605-4ff9-8567-1233be5a7bd7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.284161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716ea08-f605-4ff9-8567-1233be5a7bd7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.289206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addda6bd-02e8-4088-8fea-7eb9d88b72f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.289811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addda6bd-02e8-4088-8fea-7eb9d88b72f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.290078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addda6bd-02e8-4088-8fea-7eb9d88b72f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.290362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addda6bd-02e8-4088-8fea-7eb9d88b72f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.291172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addda6bd-02e8-4088-8fea-7eb9d88b72f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.292729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addda6bd-02e8-4088-8fea-7eb9d88b72f8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.305068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1048/2094] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:49.309383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941d29b-1381-4515-9ae9-dcee73ae6ca6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.320689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941d29b-1381-4515-9ae9-dcee73ae6ca6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.325687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb4073-5561-4668-ab89-a271f2cb72f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.326440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb4073-5561-4668-ab89-a271f2cb72f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.326836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb4073-5561-4668-ab89-a271f2cb72f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.327205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb4073-5561-4668-ab89-a271f2cb72f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.328016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb4073-5561-4668-ab89-a271f2cb72f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.329748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb4073-5561-4668-ab89-a271f2cb72f4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.340553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1047/2095] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:49.345514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addda6bd-02e8-4088-8fea-7eb9d88b72f8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.356044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addda6bd-02e8-4088-8fea-7eb9d88b72f8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.362066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc293a2-e55b-48b0-9953-94472befbce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.363077 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc293a2-e55b-48b0-9953-94472befbce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.363367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc293a2-e55b-48b0-9953-94472befbce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.363600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc293a2-e55b-48b0-9953-94472befbce8 admin admin] PATH_INFO: `/v3/projects/58a303751cfb43b6ab379efdad8b3207/users/41012bc321e643359394d2ddcc8f0692/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.379619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc293a2-e55b-48b0-9953-94472befbce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a303751cfb43b6ab379efdad8b3207, user_id=41012bc321e643359394d2ddcc8f0692, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.382302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc293a2-e55b-48b0-9953-94472befbce8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.395478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1049/2096] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:49 2022] PUT /identity/v3/projects/58a303751cfb43b6ab379efdad8b3207/users/41012bc321e643359394d2ddcc8f0692/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:49.407474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7ab88-7cb1-4560-9fc6-50b1eb0e842b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.407765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7ab88-7cb1-4560-9fc6-50b1eb0e842b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.408050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7ab88-7cb1-4560-9fc6-50b1eb0e842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.434473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2cf7ab88-7cb1-4560-9fc6-50b1eb0e842b None None] MFA Rules not processed for user `41012bc321e643359394d2ddcc8f0692`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:49.435881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2cf7ab88-7cb1-4560-9fc6-50b1eb0e842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:49.502750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1048/2097] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:49.507327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc293a2-e55b-48b0-9953-94472befbce8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.571079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc293a2-e55b-48b0-9953-94472befbce8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.576281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998f9e2-8be9-4497-807f-8b09bfcc8481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.577556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998f9e2-8be9-4497-807f-8b09bfcc8481 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.577844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998f9e2-8be9-4497-807f-8b09bfcc8481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.578201 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998f9e2-8be9-4497-807f-8b09bfcc8481 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.578926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998f9e2-8be9-4497-807f-8b09bfcc8481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.581023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998f9e2-8be9-4497-807f-8b09bfcc8481 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.598167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:49.615193 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1050/2098] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:49 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:49.619303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7ab88-7cb1-4560-9fc6-50b1eb0e842b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.634025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7ab88-7cb1-4560-9fc6-50b1eb0e842b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.710215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ad403-2b15-4228-964a-f015ccab2cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.710696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ad403-2b15-4228-964a-f015ccab2cab admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.711037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ad403-2b15-4228-964a-f015ccab2cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.711322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ad403-2b15-4228-964a-f015ccab2cab admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.712026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ad403-2b15-4228-964a-f015ccab2cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.714790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ad403-2b15-4228-964a-f015ccab2cab admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.738539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1049/2099] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:49 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:04:49.739905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998f9e2-8be9-4497-807f-8b09bfcc8481 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.752729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998f9e2-8be9-4497-807f-8b09bfcc8481 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.757615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e28324-a967-46ed-875e-e123a271ac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.758371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e28324-a967-46ed-875e-e123a271ac9f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.758729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e28324-a967-46ed-875e-e123a271ac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.759070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e28324-a967-46ed-875e-e123a271ac9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.759900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e28324-a967-46ed-875e-e123a271ac9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.761611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e28324-a967-46ed-875e-e123a271ac9f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.773302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1051/2100] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:49.777460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ad403-2b15-4228-964a-f015ccab2cab admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.787505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ad403-2b15-4228-964a-f015ccab2cab admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.792088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269eea8-ff26-46ae-86b9-900e6c6104e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.792951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269eea8-ff26-46ae-86b9-900e6c6104e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.793261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269eea8-ff26-46ae-86b9-900e6c6104e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.793536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269eea8-ff26-46ae-86b9-900e6c6104e7 admin admin] PATH_INFO: `/v3/projects/5e2f43ab41e34de280fd8276ca3be052/users/45f598a608564a23a3c0225721514d6d/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.810499 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269eea8-ff26-46ae-86b9-900e6c6104e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e2f43ab41e34de280fd8276ca3be052, user_id=45f598a608564a23a3c0225721514d6d, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.813767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269eea8-ff26-46ae-86b9-900e6c6104e7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.827757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1050/2101] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:49 2022] PUT /identity/v3/projects/5e2f43ab41e34de280fd8276ca3be052/users/45f598a608564a23a3c0225721514d6d/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:04:49.832218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e28324-a967-46ed-875e-e123a271ac9f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.842415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e28324-a967-46ed-875e-e123a271ac9f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.901691 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c3f70-e1b5-419a-a8ad-5bb7196f75b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.902283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c3f70-e1b5-419a-a8ad-5bb7196f75b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.902568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c3f70-e1b5-419a-a8ad-5bb7196f75b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.902898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c3f70-e1b5-419a-a8ad-5bb7196f75b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.903628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c3f70-e1b5-419a-a8ad-5bb7196f75b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.905251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c3f70-e1b5-419a-a8ad-5bb7196f75b1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.915310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1052/2102] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:49.920190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269eea8-ff26-46ae-86b9-900e6c6104e7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:49.927215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:49.931776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269eea8-ff26-46ae-86b9-900e6c6104e7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:49.938209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff72311-e8b3-4252-aece-09a6bdefdd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:49.939099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff72311-e8b3-4252-aece-09a6bdefdd87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.939388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff72311-e8b3-4252-aece-09a6bdefdd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.939670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff72311-e8b3-4252-aece-09a6bdefdd87 admin admin] PATH_INFO: `/v3/projects/5e2f43ab41e34de280fd8276ca3be052/users/45f598a608564a23a3c0225721514d6d/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.941968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff72311-e8b3-4252-aece-09a6bdefdd87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e2f43ab41e34de280fd8276ca3be052, user_id=45f598a608564a23a3c0225721514d6d, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:49.944672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff72311-e8b3-4252-aece-09a6bdefdd87 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:49.956657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1051/2103] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:49 2022] PUT /identity/v3/projects/5e2f43ab41e34de280fd8276ca3be052/users/45f598a608564a23a3c0225721514d6d/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:49.966872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e049a5e-4032-4cfd-94d3-3efc92e0c82b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:49.967173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e049a5e-4032-4cfd-94d3-3efc92e0c82b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:49.967493 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e049a5e-4032-4cfd-94d3-3efc92e0c82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:49.994567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7e049a5e-4032-4cfd-94d3-3efc92e0c82b None None] MFA Rules not processed for user `45f598a608564a23a3c0225721514d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:49.995812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7e049a5e-4032-4cfd-94d3-3efc92e0c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:50.066456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1053/2104] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:50.077710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff72311-e8b3-4252-aece-09a6bdefdd87 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:50.083650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-eff72311-e8b3-4252-aece-09a6bdefdd87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:50.152779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff72311-e8b3-4252-aece-09a6bdefdd87 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:50.156340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c18264-3995-47d0-b7f8-383e8932f6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:50.156986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18264-3995-47d0-b7f8-383e8932f6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:50.157373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18264-3995-47d0-b7f8-383e8932f6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:50.157661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18264-3995-47d0-b7f8-383e8932f6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:50.166920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c18264-3995-47d0-b7f8-383e8932f6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:50.168358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c18264-3995-47d0-b7f8-383e8932f6c2 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:50.179691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1052/2105] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:50.699270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e049a5e-4032-4cfd-94d3-3efc92e0c82b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:50.700732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7e049a5e-4032-4cfd-94d3-3efc92e0c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:50.714027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e049a5e-4032-4cfd-94d3-3efc92e0c82b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:50.718016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6bcd2-87b8-4326-8797-f1a228035633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:50.718690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6bcd2-87b8-4326-8797-f1a228035633 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:50.718982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6bcd2-87b8-4326-8797-f1a228035633 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:50.719260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6bcd2-87b8-4326-8797-f1a228035633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:50.784694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6bcd2-87b8-4326-8797-f1a228035633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:50.785726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6bcd2-87b8-4326-8797-f1a228035633 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:50.795385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1054/2106] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:57.021688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c18264-3995-47d0-b7f8-383e8932f6c2 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:57.032786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c18264-3995-47d0-b7f8-383e8932f6c2 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:57.038436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71aff-abf2-43a4-ac45-7765aae25c3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:57.039205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71aff-abf2-43a4-ac45-7765aae25c3e service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:57.039562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71aff-abf2-43a4-ac45-7765aae25c3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:57.039917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71aff-abf2-43a4-ac45-7765aae25c3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:57.115151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb71aff-abf2-43a4-ac45-7765aae25c3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:57.116418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb71aff-abf2-43a4-ac45-7765aae25c3e service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:57.126969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1053/2107] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:57.935191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6bcd2-87b8-4326-8797-f1a228035633 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:57.937274 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-54a6bcd2-87b8-4326-8797-f1a228035633 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:57.943996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:58.003458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6bcd2-87b8-4326-8797-f1a228035633 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:58.008003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:58.008858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:58.009176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:58.009460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] PATH_INFO: `/v3/users/ce129f79120642fc9614cbf43b38bc0f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:58.010663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce129f79120642fc9614cbf43b38bc0f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:58.012579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:58.064511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce129f79120642fc9614cbf43b38bc0f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.085383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce129f79120642fc9614cbf43b38bc0f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.094632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce129f79120642fc9614cbf43b38bc0f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.095333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce129f79120642fc9614cbf43b38bc0f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.103337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce129f79120642fc9614cbf43b38bc0f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.111445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1055/2108] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:57 2022] DELETE /identity/v3/users/ce129f79120642fc9614cbf43b38bc0f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:58.116008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbbbd9-fbff-47cc-a926-64cad075a139 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:58.116518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbbbd9-fbff-47cc-a926-64cad075a139 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:58.116633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbbbd9-fbff-47cc-a926-64cad075a139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:58.133981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-82dbbbd9-fbff-47cc-a926-64cad075a139 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:58.135195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-82dbbbd9-fbff-47cc-a926-64cad075a139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:58.208388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1054/2109] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:58.218997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:58.220440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:58.283889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d635b-75e6-472b-a2f6-8f9a342698df admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:58.287693 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3da65-86b8-4a65-a2f8-640d30c4443f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:58.288344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3da65-86b8-4a65-a2f8-640d30c4443f service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:58.288632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3da65-86b8-4a65-a2f8-640d30c4443f service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:58.288914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3da65-86b8-4a65-a2f8-640d30c4443f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:58.299237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3da65-86b8-4a65-a2f8-640d30c4443f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:58.300749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3da65-86b8-4a65-a2f8-640d30c4443f service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:58.311018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1056/2110] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:04:58.616395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbbbd9-fbff-47cc-a926-64cad075a139 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:58.618305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-82dbbbd9-fbff-47cc-a926-64cad075a139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:58.631662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbbbd9-fbff-47cc-a926-64cad075a139 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:58.636583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:58.643088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:58.646897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:58.647216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] PATH_INFO: `/v3/projects/2bb5c29776e042f0b7469529d2b4ce34` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:58.648850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb5c29776e042f0b7469529d2b4ce34)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:58.650920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:58.687455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:58.709238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb5c29776e042f0b7469529d2b4ce34', 'request_id': 'req-2cf36184-effb-48dc-8791-9d6872b9ea06'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.715785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb5c29776e042f0b7469529d2b4ce34', 'request_id': 'req-2cf36184-effb-48dc-8791-9d6872b9ea06'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.726004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb5c29776e042f0b7469529d2b4ce34', 'request_id': 'req-2cf36184-effb-48dc-8791-9d6872b9ea06'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.737005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] The token cache is being invalidate because project 2bb5c29776e042f0b7469529d2b4ce34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:04:58.737326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.737987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2cf36184-effb-48dc-8791-9d6872b9ea06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:58.739550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1055/2111] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:58 2022] DELETE /identity/v3/projects/2bb5c29776e042f0b7469529d2b4ce34 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:58.744889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c0261-a73a-42bc-ba2b-20a4d8660b60 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:58.745230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c0261-a73a-42bc-ba2b-20a4d8660b60 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:58.745558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c0261-a73a-42bc-ba2b-20a4d8660b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:58.766149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-778c0261-a73a-42bc-ba2b-20a4d8660b60 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:58.767381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-778c0261-a73a-42bc-ba2b-20a4d8660b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:58.788752 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:58.831301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1057/2112] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:58.839415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c7c1f-9077-447a-9dad-d9486817a638 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:58.839706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c7c1f-9077-447a-9dad-d9486817a638 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:58.839990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c7c1f-9077-447a-9dad-d9486817a638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:58.858958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c34c7c1f-9077-447a-9dad-d9486817a638 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:58.860246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c34c7c1f-9077-447a-9dad-d9486817a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:58.868728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1056/2113] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:04:58.872923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c0261-a73a-42bc-ba2b-20a4d8660b60 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:58.883506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c0261-a73a-42bc-ba2b-20a4d8660b60 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:58.889529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1103ba-c1ee-41e4-a496-852d431eb6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:58.890194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1103ba-c1ee-41e4-a496-852d431eb6af admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:58.890833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1103ba-c1ee-41e4-a496-852d431eb6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:58.891114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1103ba-c1ee-41e4-a496-852d431eb6af admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:58.891884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1103ba-c1ee-41e4-a496-852d431eb6af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:04:58.892203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1103ba-c1ee-41e4-a496-852d431eb6af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:58.893834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1103ba-c1ee-41e4-a496-852d431eb6af admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:58.905686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1058/2114] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:04:58.910213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c7c1f-9077-447a-9dad-d9486817a638 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:58.921724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c7c1f-9077-447a-9dad-d9486817a638 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:58.926592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0746234-6e6f-4079-935c-cdf107c88c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:58.927281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0746234-6e6f-4079-935c-cdf107c88c5a admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:58.927517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0746234-6e6f-4079-935c-cdf107c88c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:58.927928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0746234-6e6f-4079-935c-cdf107c88c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:58.928643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0746234-6e6f-4079-935c-cdf107c88c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:58.930588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0746234-6e6f-4079-935c-cdf107c88c5a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:58.958599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1057/2115] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:58 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:58.963023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1103ba-c1ee-41e4-a496-852d431eb6af admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:58.973025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1103ba-c1ee-41e4-a496-852d431eb6af admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.035757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec516da0-d035-4dcc-be78-00fe99d94cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.036555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec516da0-d035-4dcc-be78-00fe99d94cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.036840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec516da0-d035-4dcc-be78-00fe99d94cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.037151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec516da0-d035-4dcc-be78-00fe99d94cfd admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.038255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec516da0-d035-4dcc-be78-00fe99d94cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.040283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec516da0-d035-4dcc-be78-00fe99d94cfd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.064053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1059/2116] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:58 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:59.065342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0746234-6e6f-4079-935c-cdf107c88c5a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.079463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0746234-6e6f-4079-935c-cdf107c88c5a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.084118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638786e-628e-4677-8a4d-fd54ee257fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.084794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638786e-628e-4677-8a4d-fd54ee257fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.085079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638786e-628e-4677-8a4d-fd54ee257fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.085389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638786e-628e-4677-8a4d-fd54ee257fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.086122 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638786e-628e-4677-8a4d-fd54ee257fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.087726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638786e-628e-4677-8a4d-fd54ee257fd7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.099603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1058/2117] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:04:59.103589 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec516da0-d035-4dcc-be78-00fe99d94cfd admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.113804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec516da0-d035-4dcc-be78-00fe99d94cfd admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.118664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992a4c7-d12d-4c7a-a22f-7668936f39f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.119329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a4c7-d12d-4c7a-a22f-7668936f39f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.119610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a4c7-d12d-4c7a-a22f-7668936f39f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.119892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a4c7-d12d-4c7a-a22f-7668936f39f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.120615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992a4c7-d12d-4c7a-a22f-7668936f39f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.122236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992a4c7-d12d-4c7a-a22f-7668936f39f2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.137189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1060/2118] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:59.138665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638786e-628e-4677-8a4d-fd54ee257fd7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.150953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638786e-628e-4677-8a4d-fd54ee257fd7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.156154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653cdca-175f-494d-9803-4465dd6144f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.157111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653cdca-175f-494d-9803-4465dd6144f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.157466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653cdca-175f-494d-9803-4465dd6144f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.157696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653cdca-175f-494d-9803-4465dd6144f0 admin admin] PATH_INFO: `/v3/projects/7a3c0b263cbd49039b84bc5cb7bede50/users/ec1aecc28ad94ccd99ae85e31aa35f43/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.177517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653cdca-175f-494d-9803-4465dd6144f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a3c0b263cbd49039b84bc5cb7bede50, user_id=ec1aecc28ad94ccd99ae85e31aa35f43, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.180423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653cdca-175f-494d-9803-4465dd6144f0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.197022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1059/2119] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:59 2022] PUT /identity/v3/projects/7a3c0b263cbd49039b84bc5cb7bede50/users/ec1aecc28ad94ccd99ae85e31aa35f43/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:59.209506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65bb77-0582-4b33-aea7-07d20f2c49de None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.209747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65bb77-0582-4b33-aea7-07d20f2c49de None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.210038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65bb77-0582-4b33-aea7-07d20f2c49de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.241699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7e65bb77-0582-4b33-aea7-07d20f2c49de None None] MFA Rules not processed for user `ec1aecc28ad94ccd99ae85e31aa35f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:59.243108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7e65bb77-0582-4b33-aea7-07d20f2c49de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:59.305605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653cdca-175f-494d-9803-4465dd6144f0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.307660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7653cdca-175f-494d-9803-4465dd6144f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:59.334292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1061/2120] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:59.339163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65bb77-0582-4b33-aea7-07d20f2c49de None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.365396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:59.382958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653cdca-175f-494d-9803-4465dd6144f0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.387851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.388756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.389052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.389381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] PATH_INFO: `/v3/users/4fd6674f93324d779a42763e361ad124` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.390620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd6674f93324d779a42763e361ad124)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.392543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.408167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65bb77-0582-4b33-aea7-07d20f2c49de None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.412788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91c91d-1460-47b1-b1aa-f0f98eaf9f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.413594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91c91d-1460-47b1-b1aa-f0f98eaf9f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.413772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91c91d-1460-47b1-b1aa-f0f98eaf9f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.414048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91c91d-1460-47b1-b1aa-f0f98eaf9f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.414741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91c91d-1460-47b1-b1aa-f0f98eaf9f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.416690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91c91d-1460-47b1-b1aa-f0f98eaf9f64 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.443808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd6674f93324d779a42763e361ad124'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:59.446024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1062/2121] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:59 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:59.450168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91c91d-1460-47b1-b1aa-f0f98eaf9f64 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.452760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:59.462555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd6674f93324d779a42763e361ad124'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:59.470989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd6674f93324d779a42763e361ad124'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:59.480108 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd6674f93324d779a42763e361ad124'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:59.480839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3c177ea0-420e-494c-8a89-35e74a526398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd6674f93324d779a42763e361ad124'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:04:59.487706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1060/2122] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:04:59 2022] DELETE /identity/v3/users/4fd6674f93324d779a42763e361ad124 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:04:59.492176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9128-7cc2-42d4-abd9-d290363164cd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.492465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9128-7cc2-42d4-abd9-d290363164cd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.492743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9128-7cc2-42d4-abd9-d290363164cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.511515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-bcda9128-7cc2-42d4-abd9-d290363164cd None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:59.512786 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bcda9128-7cc2-42d4-abd9-d290363164cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:59.519772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91c91d-1460-47b1-b1aa-f0f98eaf9f64 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.578388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1061/2123] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:59.589688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda9128-7cc2-42d4-abd9-d290363164cd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.591171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bcda9128-7cc2-42d4-abd9-d290363164cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:59.591482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e48b09-3d7d-4988-86c5-db22ff64e819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.592682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48b09-3d7d-4988-86c5-db22ff64e819 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.592968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48b09-3d7d-4988-86c5-db22ff64e819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.593356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48b09-3d7d-4988-86c5-db22ff64e819 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.593996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e48b09-3d7d-4988-86c5-db22ff64e819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.595938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e48b09-3d7d-4988-86c5-db22ff64e819 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.626781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1063/2124] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:04:59 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:04:59.630527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e48b09-3d7d-4988-86c5-db22ff64e819 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.632246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-01e48b09-3d7d-4988-86c5-db22ff64e819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:59.644186 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e48b09-3d7d-4988-86c5-db22ff64e819 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.651487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb007b-1ed4-4719-b3cf-c25109f2df0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.652597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb007b-1ed4-4719-b3cf-c25109f2df0e admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.653047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb007b-1ed4-4719-b3cf-c25109f2df0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.653594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb007b-1ed4-4719-b3cf-c25109f2df0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.654903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda9128-7cc2-42d4-abd9-d290363164cd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.655484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb007b-1ed4-4719-b3cf-c25109f2df0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.658532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bdd84-f9a9-46fe-9935-e57de3339fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.658661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb007b-1ed4-4719-b3cf-c25109f2df0e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.659172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bdd84-f9a9-46fe-9935-e57de3339fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.659452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bdd84-f9a9-46fe-9935-e57de3339fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.659728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bdd84-f9a9-46fe-9935-e57de3339fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.669266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bdd84-f9a9-46fe-9935-e57de3339fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.671350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bdd84-f9a9-46fe-9935-e57de3339fb6 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.673592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1064/2125] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:59.677488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb007b-1ed4-4719-b3cf-c25109f2df0e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.683295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1062/2126] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:04:59.689240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb007b-1ed4-4719-b3cf-c25109f2df0e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.694279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450b0e3-e9c3-4c0c-a4ae-b1af3edf8ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.694974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450b0e3-e9c3-4c0c-a4ae-b1af3edf8ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.695262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450b0e3-e9c3-4c0c-a4ae-b1af3edf8ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.695542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450b0e3-e9c3-4c0c-a4ae-b1af3edf8ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.696268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450b0e3-e9c3-4c0c-a4ae-b1af3edf8ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.697902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450b0e3-e9c3-4c0c-a4ae-b1af3edf8ff6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.709423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1065/2127] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:04:59.713846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bdd84-f9a9-46fe-9935-e57de3339fb6 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.725912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bdd84-f9a9-46fe-9935-e57de3339fb6 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.731568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e56b9-11bd-4924-b987-5781f64510bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.733416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e56b9-11bd-4924-b987-5781f64510bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.733697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e56b9-11bd-4924-b987-5781f64510bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.733981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e56b9-11bd-4924-b987-5781f64510bb admin admin] PATH_INFO: `/v3/projects/d9812876781a417fba6f0cdfc8228125/users/4dcc57d763a94d0b8ca3893fcbc7ef4b/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.752671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e56b9-11bd-4924-b987-5781f64510bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9812876781a417fba6f0cdfc8228125, user_id=4dcc57d763a94d0b8ca3893fcbc7ef4b, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.755628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e56b9-11bd-4924-b987-5781f64510bb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.774595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1063/2128] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:04:59 2022] PUT /identity/v3/projects/d9812876781a417fba6f0cdfc8228125/users/4dcc57d763a94d0b8ca3893fcbc7ef4b/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:04:59.785878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c398b-9db6-4cc8-8132-318cf03e1bc2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.786175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c398b-9db6-4cc8-8132-318cf03e1bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.786458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c398b-9db6-4cc8-8132-318cf03e1bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.819065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-183c398b-9db6-4cc8-8132-318cf03e1bc2 None None] MFA Rules not processed for user `4dcc57d763a94d0b8ca3893fcbc7ef4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:04:59.820538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-183c398b-9db6-4cc8-8132-318cf03e1bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:04:59.832160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:59.900992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1066/2129] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:04:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:04:59.906425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e56b9-11bd-4924-b987-5781f64510bb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:04:59.966564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:04:59.983682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e56b9-11bd-4924-b987-5781f64510bb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:04:59.989025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399ccbc-6ccf-4767-9185-698bf0188ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:04:59.989766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399ccbc-6ccf-4767-9185-698bf0188ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:04:59.990091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399ccbc-6ccf-4767-9185-698bf0188ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:04:59.990378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399ccbc-6ccf-4767-9185-698bf0188ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:04:59.991134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399ccbc-6ccf-4767-9185-698bf0188ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:04:59.993176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399ccbc-6ccf-4767-9185-698bf0188ac9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:04:59.999396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c398b-9db6-4cc8-8132-318cf03e1bc2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.001270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-183c398b-9db6-4cc8-8132-318cf03e1bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:00.023065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c398b-9db6-4cc8-8132-318cf03e1bc2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.027954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.029207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1064/2130] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:04:59 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:00.029411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.029411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.029411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] PATH_INFO: `/v3/projects/c320fb4b0cb0466594a57f59a295d481` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.031089 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c320fb4b0cb0466594a57f59a295d481)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.033528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:00.034315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399ccbc-6ccf-4767-9185-698bf0188ac9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.084288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c320fb4b0cb0466594a57f59a295d481', 'request_id': 'req-ccc43796-ab52-4182-a6ed-858e44429c82'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.091087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c320fb4b0cb0466594a57f59a295d481', 'request_id': 'req-ccc43796-ab52-4182-a6ed-858e44429c82'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.100939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c320fb4b0cb0466594a57f59a295d481', 'request_id': 'req-ccc43796-ab52-4182-a6ed-858e44429c82'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.103429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399ccbc-6ccf-4767-9185-698bf0188ac9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.124280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] The token cache is being invalidate because project c320fb4b0cb0466594a57f59a295d481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:00.124960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.125692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.127273 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1067/2131] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:04:59 2022] DELETE /identity/v3/projects/c320fb4b0cb0466594a57f59a295d481 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:00.131107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.132511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:00.169375 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf5cdd-9f36-403f-a478-a78f36120756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.170834 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf5cdd-9f36-403f-a478-a78f36120756 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.171126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf5cdd-9f36-403f-a478-a78f36120756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.171415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf5cdd-9f36-403f-a478-a78f36120756 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.172119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf5cdd-9f36-403f-a478-a78f36120756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.174149 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf5cdd-9f36-403f-a478-a78f36120756 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:00.200763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1065/2132] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:00 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:05:00.204837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf5cdd-9f36-403f-a478-a78f36120756 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.206681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c0bf5cdd-9f36-403f-a478-a78f36120756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:00.208213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc43796-ab52-4182-a6ed-858e44429c82 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.214542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.215595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.215882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.216195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] PATH_INFO: `/v3/users/41012bc321e643359394d2ddcc8f0692` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.218046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41012bc321e643359394d2ddcc8f0692)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.220178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:00.221998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf5cdd-9f36-403f-a478-a78f36120756 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.227823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86577e22-068d-438c-b480-36db54161643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.228569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86577e22-068d-438c-b480-36db54161643 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.228858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86577e22-068d-438c-b480-36db54161643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.229184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86577e22-068d-438c-b480-36db54161643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.229915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86577e22-068d-438c-b480-36db54161643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.231682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86577e22-068d-438c-b480-36db54161643 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:00.249425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1066/2133] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:00.250693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86577e22-068d-438c-b480-36db54161643 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.263495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86577e22-068d-438c-b480-36db54161643 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.268439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168578ce-7cca-4680-a8c9-312710d1ec72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.268617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:00.269433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168578ce-7cca-4680-a8c9-312710d1ec72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.269720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168578ce-7cca-4680-a8c9-312710d1ec72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.269945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168578ce-7cca-4680-a8c9-312710d1ec72 admin admin] PATH_INFO: `/v3/projects/ee481d2bafdb4326af1065be59d9ce03/users/abff2b523dd04df0b5d489ddf22db612/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.281945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41012bc321e643359394d2ddcc8f0692'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.287274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168578ce-7cca-4680-a8c9-312710d1ec72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee481d2bafdb4326af1065be59d9ce03, user_id=abff2b523dd04df0b5d489ddf22db612, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.289991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168578ce-7cca-4680-a8c9-312710d1ec72 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:00.296337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41012bc321e643359394d2ddcc8f0692'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.305479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41012bc321e643359394d2ddcc8f0692'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.306165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41012bc321e643359394d2ddcc8f0692'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.306487 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1067/2134] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:00 2022] PUT /identity/v3/projects/ee481d2bafdb4326af1065be59d9ce03/users/abff2b523dd04df0b5d489ddf22db612/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:00.310883 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168578ce-7cca-4680-a8c9-312710d1ec72 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.312606 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-168578ce-7cca-4680-a8c9-312710d1ec72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:00.316920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41012bc321e643359394d2ddcc8f0692'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.331099 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1068/2135] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:00 2022] DELETE /identity/v3/users/41012bc321e643359394d2ddcc8f0692 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:00.362940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:00.388513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168578ce-7cca-4680-a8c9-312710d1ec72 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.454982 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18fb83-6184-4916-af33-2202481a2147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.455656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18fb83-6184-4916-af33-2202481a2147 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.455936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18fb83-6184-4916-af33-2202481a2147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.456209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18fb83-6184-4916-af33-2202481a2147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.456935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18fb83-6184-4916-af33-2202481a2147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.458584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18fb83-6184-4916-af33-2202481a2147 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:00.469520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1068/2136] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:05:00.474010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.484950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7a73b-8d44-4549-889f-9beee53e5392 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.491625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a5ff-8d25-40de-a5a4-c1113cb3e5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.492241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a5ff-8d25-40de-a5a4-c1113cb3e5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.492867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a5ff-8d25-40de-a5a4-c1113cb3e5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.492867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a5ff-8d25-40de-a5a4-c1113cb3e5d7 admin admin] PATH_INFO: `/v3/projects/ee481d2bafdb4326af1065be59d9ce03/users/abff2b523dd04df0b5d489ddf22db612/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.495539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06a5ff-8d25-40de-a5a4-c1113cb3e5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee481d2bafdb4326af1065be59d9ce03, user_id=abff2b523dd04df0b5d489ddf22db612, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.498489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06a5ff-8d25-40de-a5a4-c1113cb3e5d7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:00.512909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1069/2137] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:00 2022] PUT /identity/v3/projects/ee481d2bafdb4326af1065be59d9ce03/users/abff2b523dd04df0b5d489ddf22db612/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:00.524393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8100c9-3574-4567-b5ab-da1a7596af08 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.524688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8100c9-3574-4567-b5ab-da1a7596af08 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.524972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8100c9-3574-4567-b5ab-da1a7596af08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.559850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1f8100c9-3574-4567-b5ab-da1a7596af08 None None] MFA Rules not processed for user `abff2b523dd04df0b5d489ddf22db612`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:00.561620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1f8100c9-3574-4567-b5ab-da1a7596af08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:00.640292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a5ff-8d25-40de-a5a4-c1113cb3e5d7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.642066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cd06a5ff-8d25-40de-a5a4-c1113cb3e5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:00.658409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1069/2138] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:00.670211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8100c9-3574-4567-b5ab-da1a7596af08 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.671428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1f8100c9-3574-4567-b5ab-da1a7596af08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:00.740737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8100c9-3574-4567-b5ab-da1a7596af08 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.743483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a5ff-8d25-40de-a5a4-c1113cb3e5d7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.746524 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d76151-3223-47cd-90e1-9cd08b3705cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.747183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d76151-3223-47cd-90e1-9cd08b3705cc service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.747470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d76151-3223-47cd-90e1-9cd08b3705cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.747752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d76151-3223-47cd-90e1-9cd08b3705cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.750536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-05d76151-3223-47cd-90e1-9cd08b3705cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:00.751127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.751968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.752253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.752596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] PATH_INFO: `/v3/projects/58a303751cfb43b6ab379efdad8b3207` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.754041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a303751cfb43b6ab379efdad8b3207)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.755684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:00.776111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:00.818430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a303751cfb43b6ab379efdad8b3207', 'request_id': 'req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.827526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a303751cfb43b6ab379efdad8b3207', 'request_id': 'req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.836383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d76151-3223-47cd-90e1-9cd08b3705cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.838119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d76151-3223-47cd-90e1-9cd08b3705cc service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:00.839798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a303751cfb43b6ab379efdad8b3207', 'request_id': 'req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.849351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1070/2139] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:05:00.857318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] The token cache is being invalidate because project 58a303751cfb43b6ab379efdad8b3207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:00.858148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.859059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:00.864040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1070/2140] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:00 2022] DELETE /identity/v3/projects/58a303751cfb43b6ab379efdad8b3207 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:00.865969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d76151-3223-47cd-90e1-9cd08b3705cc service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:00.878495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d76151-3223-47cd-90e1-9cd08b3705cc service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:00.959580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:00.961037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:00.961632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:00.962498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] PATH_INFO: `/v3/users/45f598a608564a23a3c0225721514d6d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:00.965654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f598a608564a23a3c0225721514d6d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:00.968652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:01.058484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f598a608564a23a3c0225721514d6d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.079940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f598a608564a23a3c0225721514d6d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.088809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f598a608564a23a3c0225721514d6d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.097821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f598a608564a23a3c0225721514d6d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.098544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b52b276f-dbd7-4338-8104-6b948264076d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f598a608564a23a3c0225721514d6d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.107913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1071/2141] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:00 2022] DELETE /identity/v3/users/45f598a608564a23a3c0225721514d6d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:01.656736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:01.658302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:01.671632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db7560-b6e8-4e40-8eef-d20d24d0cbdc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:01.709795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:01.744101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:01.745039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:01.745299 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:01.745583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] PATH_INFO: `/v3/projects/5e2f43ab41e34de280fd8276ca3be052` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:01.746781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e2f43ab41e34de280fd8276ca3be052)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:01.749311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:01.798582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e2f43ab41e34de280fd8276ca3be052', 'request_id': 'req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.804887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e2f43ab41e34de280fd8276ca3be052', 'request_id': 'req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.813445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e2f43ab41e34de280fd8276ca3be052', 'request_id': 'req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.824922 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] The token cache is being invalidate because project 5e2f43ab41e34de280fd8276ca3be052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:01.825242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.825892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2d25f4ac-20c8-4cf6-9ff0-ecf573ecb3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:01.828023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1071/2142] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:01 2022] DELETE /identity/v3/projects/5e2f43ab41e34de280fd8276ca3be052 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:01.833963 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0d003-717f-4315-89d9-2c282bfbc6d1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:01.834470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0d003-717f-4315-89d9-2c282bfbc6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:01.834555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0d003-717f-4315-89d9-2c282bfbc6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:01.854099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ecf0d003-717f-4315-89d9-2c282bfbc6d1 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:01.855650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ecf0d003-717f-4315-89d9-2c282bfbc6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:01.891404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:01.918233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1072/2143] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:01.927245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2841a2c-b3c5-4523-a421-382f28387572 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:01.927537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2841a2c-b3c5-4523-a421-382f28387572 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:01.927819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2841a2c-b3c5-4523-a421-382f28387572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:01.946496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f2841a2c-b3c5-4523-a421-382f28387572 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:01.947716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f2841a2c-b3c5-4523-a421-382f28387572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:01.955665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1072/2144] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:01.959831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0d003-717f-4315-89d9-2c282bfbc6d1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:01.970607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0d003-717f-4315-89d9-2c282bfbc6d1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:01.975175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833a3a6-e9af-4f11-8ec5-88c9741a021f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:01.975825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833a3a6-e9af-4f11-8ec5-88c9741a021f admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:01.976114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833a3a6-e9af-4f11-8ec5-88c9741a021f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:01.976399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833a3a6-e9af-4f11-8ec5-88c9741a021f admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:01.977165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833a3a6-e9af-4f11-8ec5-88c9741a021f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:05:01.977486 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833a3a6-e9af-4f11-8ec5-88c9741a021f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:01.979090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833a3a6-e9af-4f11-8ec5-88c9741a021f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:01.991304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1073/2145] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:05:01.995583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2841a2c-b3c5-4523-a421-382f28387572 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.004703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2841a2c-b3c5-4523-a421-382f28387572 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.009363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5fca-b83a-4045-81e2-6233e14ae04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.010006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5fca-b83a-4045-81e2-6233e14ae04c admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.010291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5fca-b83a-4045-81e2-6233e14ae04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.010574 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5fca-b83a-4045-81e2-6233e14ae04c admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.011268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc5fca-b83a-4045-81e2-6233e14ae04c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.013197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc5fca-b83a-4045-81e2-6233e14ae04c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.039154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1073/2146] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:01 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:02.043403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833a3a6-e9af-4f11-8ec5-88c9741a021f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.054102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833a3a6-e9af-4f11-8ec5-88c9741a021f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.120246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5383489-44a5-4f60-8023-6659b609a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.121028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5383489-44a5-4f60-8023-6659b609a2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.121480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5383489-44a5-4f60-8023-6659b609a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.121592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5383489-44a5-4f60-8023-6659b609a2f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.122298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5383489-44a5-4f60-8023-6659b609a2f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.124557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5383489-44a5-4f60-8023-6659b609a2f7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.149376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1074/2147] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:02 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:05:02.150536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5fca-b83a-4045-81e2-6233e14ae04c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.161946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5fca-b83a-4045-81e2-6233e14ae04c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.166741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020c8f3-29cb-4f55-ade1-8509478e8acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.167409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020c8f3-29cb-4f55-ade1-8509478e8acf admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.167694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020c8f3-29cb-4f55-ade1-8509478e8acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.167972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020c8f3-29cb-4f55-ade1-8509478e8acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.168923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020c8f3-29cb-4f55-ade1-8509478e8acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.170639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020c8f3-29cb-4f55-ade1-8509478e8acf admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.181663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1074/2148] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:05:02.186552 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5383489-44a5-4f60-8023-6659b609a2f7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.197466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5383489-44a5-4f60-8023-6659b609a2f7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.202614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaa426-0be2-4413-9b9c-cd8536c2b8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.203277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa426-0be2-4413-9b9c-cd8536c2b8ac admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.203567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa426-0be2-4413-9b9c-cd8536c2b8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.203848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa426-0be2-4413-9b9c-cd8536c2b8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.204583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaa426-0be2-4413-9b9c-cd8536c2b8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.206235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaa426-0be2-4413-9b9c-cd8536c2b8ac admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.216370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1075/2149] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:02.221358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020c8f3-29cb-4f55-ade1-8509478e8acf admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.232207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020c8f3-29cb-4f55-ade1-8509478e8acf admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.237276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538af74-e266-4a46-b294-0aad6ac2db41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.238198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538af74-e266-4a46-b294-0aad6ac2db41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.238486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538af74-e266-4a46-b294-0aad6ac2db41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.238764 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538af74-e266-4a46-b294-0aad6ac2db41 admin admin] PATH_INFO: `/v3/projects/f90e8f2e83874d68acc88db7b14935ef/users/b0c85e663a89489193257b35066868e2/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.255661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538af74-e266-4a46-b294-0aad6ac2db41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90e8f2e83874d68acc88db7b14935ef, user_id=b0c85e663a89489193257b35066868e2, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.258343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538af74-e266-4a46-b294-0aad6ac2db41 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.272184 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1075/2150] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:02 2022] PUT /identity/v3/projects/f90e8f2e83874d68acc88db7b14935ef/users/b0c85e663a89489193257b35066868e2/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:02.285416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f69ed7-edc9-400c-87a8-89e24efe37a0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.285711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f69ed7-edc9-400c-87a8-89e24efe37a0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.285992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f69ed7-edc9-400c-87a8-89e24efe37a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.312132 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b5f69ed7-edc9-400c-87a8-89e24efe37a0 None None] MFA Rules not processed for user `b0c85e663a89489193257b35066868e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:02.313383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b5f69ed7-edc9-400c-87a8-89e24efe37a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:02.381965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1076/2151] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:02.386424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538af74-e266-4a46-b294-0aad6ac2db41 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.447988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:02.452608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538af74-e266-4a46-b294-0aad6ac2db41 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.459018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736ab29-1839-46e7-8206-5b9256d955a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.459843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736ab29-1839-46e7-8206-5b9256d955a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.460561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736ab29-1839-46e7-8206-5b9256d955a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.460946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736ab29-1839-46e7-8206-5b9256d955a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.461694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736ab29-1839-46e7-8206-5b9256d955a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.463593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736ab29-1839-46e7-8206-5b9256d955a6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.489036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1076/2152] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:02 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:05:02.492878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f69ed7-edc9-400c-87a8-89e24efe37a0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.504642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f69ed7-edc9-400c-87a8-89e24efe37a0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.541355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:02.570951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc0237-0050-43d0-bfe1-6d52a574e69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.571782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc0237-0050-43d0-bfe1-6d52a574e69d admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.572140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc0237-0050-43d0-bfe1-6d52a574e69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.572492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc0237-0050-43d0-bfe1-6d52a574e69d admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.573317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc0237-0050-43d0-bfe1-6d52a574e69d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.575342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc0237-0050-43d0-bfe1-6d52a574e69d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.597000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1077/2153] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:02 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:02.602005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736ab29-1839-46e7-8206-5b9256d955a6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.612795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736ab29-1839-46e7-8206-5b9256d955a6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.617650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa1613-6c76-4baf-8b5c-262d01de359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.618362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa1613-6c76-4baf-8b5c-262d01de359e admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.618658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa1613-6c76-4baf-8b5c-262d01de359e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.618931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa1613-6c76-4baf-8b5c-262d01de359e admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.619666 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa1613-6c76-4baf-8b5c-262d01de359e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.621344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa1613-6c76-4baf-8b5c-262d01de359e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.633256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1077/2154] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:02.637502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc0237-0050-43d0-bfe1-6d52a574e69d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.648621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc0237-0050-43d0-bfe1-6d52a574e69d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.654209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7269ef4-b728-4e3d-b9ff-4c17c7cd3bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.655117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269ef4-b728-4e3d-b9ff-4c17c7cd3bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.655409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269ef4-b728-4e3d-b9ff-4c17c7cd3bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.655688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269ef4-b728-4e3d-b9ff-4c17c7cd3bf7 admin admin] PATH_INFO: `/v3/projects/edaa461f11354183b2538e9d86b48fab/users/9f1c41e0f33a46e083090c25d3d5bc76/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.672242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7269ef4-b728-4e3d-b9ff-4c17c7cd3bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edaa461f11354183b2538e9d86b48fab, user_id=9f1c41e0f33a46e083090c25d3d5bc76, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.675087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7269ef4-b728-4e3d-b9ff-4c17c7cd3bf7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.688874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1078/2155] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:02 2022] PUT /identity/v3/projects/edaa461f11354183b2538e9d86b48fab/users/9f1c41e0f33a46e083090c25d3d5bc76/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:02.693324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa1613-6c76-4baf-8b5c-262d01de359e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.704054 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa1613-6c76-4baf-8b5c-262d01de359e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.760865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee11ce4-2feb-4729-b2b2-0229f86a7c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.761495 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11ce4-2feb-4729-b2b2-0229f86a7c1a admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.761780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11ce4-2feb-4729-b2b2-0229f86a7c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.762053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11ce4-2feb-4729-b2b2-0229f86a7c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.762785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee11ce4-2feb-4729-b2b2-0229f86a7c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.765001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee11ce4-2feb-4729-b2b2-0229f86a7c1a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.775197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1078/2156] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:05:02.779800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7269ef4-b728-4e3d-b9ff-4c17c7cd3bf7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.790597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7269ef4-b728-4e3d-b9ff-4c17c7cd3bf7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.795373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36b35b-10c8-40f1-a54c-7a909ab21af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.796260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36b35b-10c8-40f1-a54c-7a909ab21af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.796548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36b35b-10c8-40f1-a54c-7a909ab21af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.796836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36b35b-10c8-40f1-a54c-7a909ab21af4 admin admin] PATH_INFO: `/v3/projects/edaa461f11354183b2538e9d86b48fab/users/9f1c41e0f33a46e083090c25d3d5bc76/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.798848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36b35b-10c8-40f1-a54c-7a909ab21af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edaa461f11354183b2538e9d86b48fab, user_id=9f1c41e0f33a46e083090c25d3d5bc76, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:02.801547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36b35b-10c8-40f1-a54c-7a909ab21af4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:02.812738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1079/2157] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:02 2022] PUT /identity/v3/projects/edaa461f11354183b2538e9d86b48fab/users/9f1c41e0f33a46e083090c25d3d5bc76/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:02.823351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae8953-358f-45fb-8e8c-e3ca534033fe None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.823648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae8953-358f-45fb-8e8c-e3ca534033fe None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.823959 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae8953-358f-45fb-8e8c-e3ca534033fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:02.851902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f0ae8953-358f-45fb-8e8c-e3ca534033fe None None] MFA Rules not processed for user `9f1c41e0f33a46e083090c25d3d5bc76`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:02.853143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f0ae8953-358f-45fb-8e8c-e3ca534033fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:02.923087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1079/2158] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:02.934121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36b35b-10c8-40f1-a54c-7a909ab21af4 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:02.935974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1f36b35b-10c8-40f1-a54c-7a909ab21af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:02.994260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36b35b-10c8-40f1-a54c-7a909ab21af4 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:02.997952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e5aa7-9521-4374-a392-3f64b1ca35f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:02.998608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e5aa7-9521-4374-a392-3f64b1ca35f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:02.998913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e5aa7-9521-4374-a392-3f64b1ca35f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:02.999195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e5aa7-9521-4374-a392-3f64b1ca35f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:03.008458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e5aa7-9521-4374-a392-3f64b1ca35f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:03.009945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e5aa7-9521-4374-a392-3f64b1ca35f1 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:03.020311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1080/2159] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:03.437108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae8953-358f-45fb-8e8c-e3ca534033fe None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:03.491337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:03.503968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae8953-358f-45fb-8e8c-e3ca534033fe None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:03.508494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:03.509385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:03.509665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:03.509942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] PATH_INFO: `/v3/users/b0c85e663a89489193257b35066868e2` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:03.511126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0c85e663a89489193257b35066868e2)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:03.513028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:03.564358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0c85e663a89489193257b35066868e2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:03.578832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0c85e663a89489193257b35066868e2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:03.586802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0c85e663a89489193257b35066868e2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:03.587546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0c85e663a89489193257b35066868e2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:03.595349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0c85e663a89489193257b35066868e2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:03.602396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1080/2160] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:03 2022] DELETE /identity/v3/users/b0c85e663a89489193257b35066868e2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:03.607204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f55c7-afd3-434c-901c-d9d27902f454 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:03.607551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f55c7-afd3-434c-901c-d9d27902f454 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:03.607838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f55c7-afd3-434c-901c-d9d27902f454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:03.628026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f46f55c7-afd3-434c-901c-d9d27902f454 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:03.629359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f46f55c7-afd3-434c-901c-d9d27902f454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:03.691737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1081/2161] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:03.702713 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:03.704258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:03.765140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5076053-cdee-4576-9257-e81be784a692 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:03.768830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d932a-9898-458a-a654-1c712a6f6711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:03.769516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d932a-9898-458a-a654-1c712a6f6711 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:03.769806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d932a-9898-458a-a654-1c712a6f6711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:03.770087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d932a-9898-458a-a654-1c712a6f6711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:03.780373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d932a-9898-458a-a654-1c712a6f6711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:03.781904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d932a-9898-458a-a654-1c712a6f6711 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:03.792155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1081/2162] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:04.090115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f55c7-afd3-434c-901c-d9d27902f454 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:04.092237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f46f55c7-afd3-434c-901c-d9d27902f454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:04.104429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f55c7-afd3-434c-901c-d9d27902f454 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:04.109897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:04.110802 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:04.111116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:04.111400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] PATH_INFO: `/v3/projects/f90e8f2e83874d68acc88db7b14935ef` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:04.112602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90e8f2e83874d68acc88db7b14935ef)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:04.114610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:04.145299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:04.163681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90e8f2e83874d68acc88db7b14935ef', 'request_id': 'req-1fd3a7a2-15f2-443f-9e15-d45661f05de3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.170485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90e8f2e83874d68acc88db7b14935ef', 'request_id': 'req-1fd3a7a2-15f2-443f-9e15-d45661f05de3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.178066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90e8f2e83874d68acc88db7b14935ef', 'request_id': 'req-1fd3a7a2-15f2-443f-9e15-d45661f05de3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.191248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] The token cache is being invalidate because project f90e8f2e83874d68acc88db7b14935ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:04.191543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.192198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.194067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1082/2163] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:04 2022] DELETE /identity/v3/projects/f90e8f2e83874d68acc88db7b14935ef => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:04.197788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d932a-9898-458a-a654-1c712a6f6711 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:04.207895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d932a-9898-458a-a654-1c712a6f6711 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:04.276393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:04.277288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:04.277578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:04.277858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] PATH_INFO: `/v3/users/9f1c41e0f33a46e083090c25d3d5bc76` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:04.279064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f1c41e0f33a46e083090c25d3d5bc76)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:04.281558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:04.297919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:04.335289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f1c41e0f33a46e083090c25d3d5bc76'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.348172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f1c41e0f33a46e083090c25d3d5bc76'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.356147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f1c41e0f33a46e083090c25d3d5bc76'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.356810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f1c41e0f33a46e083090c25d3d5bc76'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.364639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08b7704a-ecf6-4b6c-80ab-99b8896cfb90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f1c41e0f33a46e083090c25d3d5bc76'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.370785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1082/2164] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:04 2022] DELETE /identity/v3/users/9f1c41e0f33a46e083090c25d3d5bc76 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:04.649285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:04.650793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:04.664481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3a7a2-15f2-443f-9e15-d45661f05de3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:04.722859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:04.723727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:04.724008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:04.724290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] PATH_INFO: `/v3/projects/edaa461f11354183b2538e9d86b48fab` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:04.725498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edaa461f11354183b2538e9d86b48fab)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:04.727440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:04.772911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edaa461f11354183b2538e9d86b48fab', 'request_id': 'req-63ca9673-a306-46f4-b8a8-cb35fbb345d7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.778928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edaa461f11354183b2538e9d86b48fab', 'request_id': 'req-63ca9673-a306-46f4-b8a8-cb35fbb345d7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.786827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edaa461f11354183b2538e9d86b48fab', 'request_id': 'req-63ca9673-a306-46f4-b8a8-cb35fbb345d7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.797799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] The token cache is being invalidate because project edaa461f11354183b2538e9d86b48fab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:04.798083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.798709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-63ca9673-a306-46f4-b8a8-cb35fbb345d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:04.800250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1083/2165] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:04 2022] DELETE /identity/v3/projects/edaa461f11354183b2538e9d86b48fab => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:04.805045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b3687-245c-4df2-8e5d-5f9246b464f0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:04.805542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b3687-245c-4df2-8e5d-5f9246b464f0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:04.805832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b3687-245c-4df2-8e5d-5f9246b464f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:04.824629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e06b3687-245c-4df2-8e5d-5f9246b464f0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:04.825872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e06b3687-245c-4df2-8e5d-5f9246b464f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:04.885183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1083/2166] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:04.893888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c230e-d1a9-4b19-ac71-fe71fb66626b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:04.894213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c230e-d1a9-4b19-ac71-fe71fb66626b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:04.894452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c230e-d1a9-4b19-ac71-fe71fb66626b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:04.912855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-192c230e-d1a9-4b19-ac71-fe71fb66626b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:04.914086 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-192c230e-d1a9-4b19-ac71-fe71fb66626b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:04.922620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1084/2167] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:05:04.927465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b3687-245c-4df2-8e5d-5f9246b464f0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:04.937626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b3687-245c-4df2-8e5d-5f9246b464f0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:04.942625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b752168-5c01-47b1-90fe-510273284232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:04.943288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b752168-5c01-47b1-90fe-510273284232 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:04.943573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b752168-5c01-47b1-90fe-510273284232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:04.943886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b752168-5c01-47b1-90fe-510273284232 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:04.944622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b752168-5c01-47b1-90fe-510273284232 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:05:04.944934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b752168-5c01-47b1-90fe-510273284232 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:04.946575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b752168-5c01-47b1-90fe-510273284232 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:04.959242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1084/2168] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:05:04.965573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c230e-d1a9-4b19-ac71-fe71fb66626b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:04.976151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c230e-d1a9-4b19-ac71-fe71fb66626b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:04.984871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef4d33-c0c8-42d7-9f0b-89afc4973031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:04.985592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4d33-c0c8-42d7-9f0b-89afc4973031 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:04.985872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4d33-c0c8-42d7-9f0b-89afc4973031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:04.986149 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4d33-c0c8-42d7-9f0b-89afc4973031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:04.986852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef4d33-c0c8-42d7-9f0b-89afc4973031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:04.988772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef4d33-c0c8-42d7-9f0b-89afc4973031 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.018457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1085/2169] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:04 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:05.019666 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b752168-5c01-47b1-90fe-510273284232 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.030982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b752168-5c01-47b1-90fe-510273284232 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.087976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebca9bf-6366-49bc-9c9c-8e5d08c09ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.088645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebca9bf-6366-49bc-9c9c-8e5d08c09ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.088932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebca9bf-6366-49bc-9c9c-8e5d08c09ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.089244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebca9bf-6366-49bc-9c9c-8e5d08c09ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.089976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebca9bf-6366-49bc-9c9c-8e5d08c09ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.092459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebca9bf-6366-49bc-9c9c-8e5d08c09ed7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.112837 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1085/2170] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:05 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:05.116992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef4d33-c0c8-42d7-9f0b-89afc4973031 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.127712 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef4d33-c0c8-42d7-9f0b-89afc4973031 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.132475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e538a-0321-4da9-af84-869a9d4772f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.133173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e538a-0321-4da9-af84-869a9d4772f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.133464 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e538a-0321-4da9-af84-869a9d4772f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.133746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e538a-0321-4da9-af84-869a9d4772f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.134480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e538a-0321-4da9-af84-869a9d4772f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.136148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e538a-0321-4da9-af84-869a9d4772f5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.147269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1086/2171] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:05.151417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebca9bf-6366-49bc-9c9c-8e5d08c09ed7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.161898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebca9bf-6366-49bc-9c9c-8e5d08c09ed7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.166439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f92ca9-bec5-4cfa-af48-06020d0870b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.167083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f92ca9-bec5-4cfa-af48-06020d0870b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.167364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f92ca9-bec5-4cfa-af48-06020d0870b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.167642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f92ca9-bec5-4cfa-af48-06020d0870b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.168351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f92ca9-bec5-4cfa-af48-06020d0870b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.169976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f92ca9-bec5-4cfa-af48-06020d0870b1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.175069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:05.181081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1086/2172] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:05.185503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e538a-0321-4da9-af84-869a9d4772f5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.192300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:05.196227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e538a-0321-4da9-af84-869a9d4772f5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.203103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e14d6b-890e-4f9a-9f96-063c300def8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.204008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e14d6b-890e-4f9a-9f96-063c300def8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.204468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e14d6b-890e-4f9a-9f96-063c300def8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.204770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e14d6b-890e-4f9a-9f96-063c300def8f admin admin] PATH_INFO: `/v3/projects/25ba6da001894ab9aa80068e985e6138/users/398df5c44f264a5c800f9a06a42d8aa0/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.220479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e14d6b-890e-4f9a-9f96-063c300def8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ba6da001894ab9aa80068e985e6138, user_id=398df5c44f264a5c800f9a06a42d8aa0, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.223184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e14d6b-890e-4f9a-9f96-063c300def8f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.236071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1087/2173] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:05 2022] PUT /identity/v3/projects/25ba6da001894ab9aa80068e985e6138/users/398df5c44f264a5c800f9a06a42d8aa0/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:05.246394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d67375-d4d6-4ce2-b4f7-cd61f113c2cf None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.246613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d67375-d4d6-4ce2-b4f7-cd61f113c2cf None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.246896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d67375-d4d6-4ce2-b4f7-cd61f113c2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.275449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c8d67375-d4d6-4ce2-b4f7-cd61f113c2cf None None] MFA Rules not processed for user `398df5c44f264a5c800f9a06a42d8aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:05.276692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c8d67375-d4d6-4ce2-b4f7-cd61f113c2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:05.344500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1087/2174] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:05.349168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e14d6b-890e-4f9a-9f96-063c300def8f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.412137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e14d6b-890e-4f9a-9f96-063c300def8f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.417700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a350da8-12aa-4193-966d-f65e85a1b610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.418374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a350da8-12aa-4193-966d-f65e85a1b610 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.418664 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a350da8-12aa-4193-966d-f65e85a1b610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.418952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a350da8-12aa-4193-966d-f65e85a1b610 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.419668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a350da8-12aa-4193-966d-f65e85a1b610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.422027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a350da8-12aa-4193-966d-f65e85a1b610 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.447523 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1088/2175] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:05 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:05.452117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d67375-d4d6-4ce2-b4f7-cd61f113c2cf None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.462967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d67375-d4d6-4ce2-b4f7-cd61f113c2cf None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.520945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605a720-93fa-437b-8436-de113a0a0fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.521530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605a720-93fa-437b-8436-de113a0a0fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.522182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605a720-93fa-437b-8436-de113a0a0fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.522182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605a720-93fa-437b-8436-de113a0a0fb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.522832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605a720-93fa-437b-8436-de113a0a0fb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.524729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605a720-93fa-437b-8436-de113a0a0fb8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.544716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1088/2176] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:05 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:05.548572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a350da8-12aa-4193-966d-f65e85a1b610 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.559087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a350da8-12aa-4193-966d-f65e85a1b610 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.564192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df9e88-1eef-4494-b6e9-687f97352ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.564848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df9e88-1eef-4494-b6e9-687f97352ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.565162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df9e88-1eef-4494-b6e9-687f97352ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.565442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df9e88-1eef-4494-b6e9-687f97352ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.566427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df9e88-1eef-4494-b6e9-687f97352ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.568020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df9e88-1eef-4494-b6e9-687f97352ac8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.580135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1089/2177] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:05.584340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605a720-93fa-437b-8436-de113a0a0fb8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.594509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605a720-93fa-437b-8436-de113a0a0fb8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.599446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6833d-c472-4dc6-9758-357e3485c57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.600370 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6833d-c472-4dc6-9758-357e3485c57a admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.600656 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6833d-c472-4dc6-9758-357e3485c57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.600938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6833d-c472-4dc6-9758-357e3485c57a admin admin] PATH_INFO: `/v3/projects/dc62b1fce1cc4a488d9e440c1ba5f622/users/27d64ef09d8949579112bff7e5a9615f/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.616744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6833d-c472-4dc6-9758-357e3485c57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc62b1fce1cc4a488d9e440c1ba5f622, user_id=27d64ef09d8949579112bff7e5a9615f, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.619485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6833d-c472-4dc6-9758-357e3485c57a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.633731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1089/2178] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:05 2022] PUT /identity/v3/projects/dc62b1fce1cc4a488d9e440c1ba5f622/users/27d64ef09d8949579112bff7e5a9615f/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:05.637685 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df9e88-1eef-4494-b6e9-687f97352ac8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.648432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df9e88-1eef-4494-b6e9-687f97352ac8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.708235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafea78-05a2-4720-875b-6ddf3f3a6ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.708900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafea78-05a2-4720-875b-6ddf3f3a6ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.709212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafea78-05a2-4720-875b-6ddf3f3a6ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.709491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafea78-05a2-4720-875b-6ddf3f3a6ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.710244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafea78-05a2-4720-875b-6ddf3f3a6ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.711848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafea78-05a2-4720-875b-6ddf3f3a6ed7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.722098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1090/2179] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:05.726128 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6833d-c472-4dc6-9758-357e3485c57a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.736721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6833d-c472-4dc6-9758-357e3485c57a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.742522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd509-b4c2-45ea-a8ce-65b9c0836f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.743706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd509-b4c2-45ea-a8ce-65b9c0836f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.744071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd509-b4c2-45ea-a8ce-65b9c0836f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.744349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd509-b4c2-45ea-a8ce-65b9c0836f90 admin admin] PATH_INFO: `/v3/projects/dc62b1fce1cc4a488d9e440c1ba5f622/users/27d64ef09d8949579112bff7e5a9615f/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.746591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd509-b4c2-45ea-a8ce-65b9c0836f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc62b1fce1cc4a488d9e440c1ba5f622, user_id=27d64ef09d8949579112bff7e5a9615f, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:05.749876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd509-b4c2-45ea-a8ce-65b9c0836f90 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:05.760763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1090/2180] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:05 2022] PUT /identity/v3/projects/dc62b1fce1cc4a488d9e440c1ba5f622/users/27d64ef09d8949579112bff7e5a9615f/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:05.771965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54282d-6746-421d-a4ce-fa0850a58e94 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.772258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54282d-6746-421d-a4ce-fa0850a58e94 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.772567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54282d-6746-421d-a4ce-fa0850a58e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:05.798631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-dc54282d-6746-421d-a4ce-fa0850a58e94 None None] MFA Rules not processed for user `27d64ef09d8949579112bff7e5a9615f`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:05.799839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-dc54282d-6746-421d-a4ce-fa0850a58e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:05.841209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:05.870467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1091/2181] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:05.881712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd509-b4c2-45ea-a8ce-65b9c0836f90 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:05.883589 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7f5cd509-b4c2-45ea-a8ce-65b9c0836f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:05.930415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:05.944046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd509-b4c2-45ea-a8ce-65b9c0836f90 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:05.947777 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b56bd-f561-4133-a0b2-54f746246c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:05.948416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b56bd-f561-4133-a0b2-54f746246c8d service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:05.948701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b56bd-f561-4133-a0b2-54f746246c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:05.948982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b56bd-f561-4133-a0b2-54f746246c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:06.007721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b56bd-f561-4133-a0b2-54f746246c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:06.009192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b56bd-f561-4133-a0b2-54f746246c8d service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:06.018634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1091/2182] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:15.019382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54282d-6746-421d-a4ce-fa0850a58e94 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:15.039500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54282d-6746-421d-a4ce-fa0850a58e94 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:15.043078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738d7e5-fea5-4b14-b209-d0992d74391c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:15.043727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738d7e5-fea5-4b14-b209-d0992d74391c service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:15.044008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738d7e5-fea5-4b14-b209-d0992d74391c service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:15.044286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738d7e5-fea5-4b14-b209-d0992d74391c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:15.046210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d738d7e5-fea5-4b14-b209-d0992d74391c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:15.128341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738d7e5-fea5-4b14-b209-d0992d74391c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:15.129819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738d7e5-fea5-4b14-b209-d0992d74391c service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:15.139229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1092/2183] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:15.684494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b56bd-f561-4133-a0b2-54f746246c8d service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:15.703536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b56bd-f561-4133-a0b2-54f746246c8d service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:15.707211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:15.707874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:15.708158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:15.708438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:15.710244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:15.783408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:15.784895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:15.795017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1092/2184] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:18.261697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f13fb-7573-401a-a57e-31e17bea80b6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:18.262254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f13fb-7573-401a-a57e-31e17bea80b6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:18.262453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f13fb-7573-401a-a57e-31e17bea80b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:18.297227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2b8f13fb-7573-401a-a57e-31e17bea80b6 None None] MFA Rules not processed for user `569256e6d1d1418eafb5ff2ed8909926`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:18.299769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2b8f13fb-7573-401a-a57e-31e17bea80b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:18.392463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1093/2185] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:05:20.896316 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:20.898131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:20.923319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39528087-616c-4665-863a-c5f830a4d9ac service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:20.927574 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16aad3-fbd9-4411-805a-03e1facf3caf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:20.928261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16aad3-fbd9-4411-805a-03e1facf3caf service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:20.928547 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16aad3-fbd9-4411-805a-03e1facf3caf service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:20.928827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16aad3-fbd9-4411-805a-03e1facf3caf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:20.942312 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16aad3-fbd9-4411-805a-03e1facf3caf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:20.952991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16aad3-fbd9-4411-805a-03e1facf3caf service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:20.957381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1093/2186] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:21.046670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f13fb-7573-401a-a57e-31e17bea80b6 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:21.048438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2b8f13fb-7573-401a-a57e-31e17bea80b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:21.116588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:21.178313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f13fb-7573-401a-a57e-31e17bea80b6 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:21.188181 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:21.189293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:21.189548 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:21.189834 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] PATH_INFO: `/v3/users/ec1aecc28ad94ccd99ae85e31aa35f43` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:21.191559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec1aecc28ad94ccd99ae85e31aa35f43)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:21.194340 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:21.256453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec1aecc28ad94ccd99ae85e31aa35f43'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:21.289226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec1aecc28ad94ccd99ae85e31aa35f43'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:21.303315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec1aecc28ad94ccd99ae85e31aa35f43'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:21.321624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec1aecc28ad94ccd99ae85e31aa35f43'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:21.327276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec1aecc28ad94ccd99ae85e31aa35f43'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:21.339738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1094/2187] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:21 2022] DELETE /identity/v3/users/ec1aecc28ad94ccd99ae85e31aa35f43 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:21.346879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dae541-43f5-4180-ba08-92b42af0c3a7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:21.348636 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dae541-43f5-4180-ba08-92b42af0c3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:21.349632 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dae541-43f5-4180-ba08-92b42af0c3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:21.398468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-65dae541-43f5-4180-ba08-92b42af0c3a7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:21.399689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-65dae541-43f5-4180-ba08-92b42af0c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:21.462409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:21.492546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1094/2188] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:21.503345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:21.504822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:21.587713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24d083-5eba-4637-8fad-10283c75c484 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:21.591503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10f52-4529-4224-bf50-c5a96c1ad13f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:21.592150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10f52-4529-4224-bf50-c5a96c1ad13f service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:21.592432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10f52-4529-4224-bf50-c5a96c1ad13f service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:21.592709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10f52-4529-4224-bf50-c5a96c1ad13f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:21.607322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10f52-4529-4224-bf50-c5a96c1ad13f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:21.608645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10f52-4529-4224-bf50-c5a96c1ad13f service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:21.620442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1095/2189] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:21.924925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dae541-43f5-4180-ba08-92b42af0c3a7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:21.926450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-65dae541-43f5-4180-ba08-92b42af0c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:21.938907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dae541-43f5-4180-ba08-92b42af0c3a7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:21.944113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:21.944984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:21.945297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:21.945577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] PATH_INFO: `/v3/projects/7a3c0b263cbd49039b84bc5cb7bede50` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:21.946790 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a3c0b263cbd49039b84bc5cb7bede50)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:21.948783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:21.998256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a3c0b263cbd49039b84bc5cb7bede50', 'request_id': 'req-bbcb1164-aa01-4ff8-93dd-a598245e6453'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.004572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a3c0b263cbd49039b84bc5cb7bede50', 'request_id': 'req-bbcb1164-aa01-4ff8-93dd-a598245e6453'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.012122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a3c0b263cbd49039b84bc5cb7bede50', 'request_id': 'req-bbcb1164-aa01-4ff8-93dd-a598245e6453'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.023246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] The token cache is being invalidate because project 7a3c0b263cbd49039b84bc5cb7bede50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:22.023533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.024180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.025765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1095/2190] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:21 2022] DELETE /identity/v3/projects/7a3c0b263cbd49039b84bc5cb7bede50 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:22.029654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10f52-4529-4224-bf50-c5a96c1ad13f service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:22.040160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10f52-4529-4224-bf50-c5a96c1ad13f service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:22.061565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:22.110618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:22.111788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:22.112076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:22.112485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] PATH_INFO: `/v3/users/4dcc57d763a94d0b8ca3893fcbc7ef4b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:22.113744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dcc57d763a94d0b8ca3893fcbc7ef4b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:22.115706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:22.177505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dcc57d763a94d0b8ca3893fcbc7ef4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.191995 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dcc57d763a94d0b8ca3893fcbc7ef4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.201207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dcc57d763a94d0b8ca3893fcbc7ef4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.208539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dcc57d763a94d0b8ca3893fcbc7ef4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.209275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dcc57d763a94d0b8ca3893fcbc7ef4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:22.215599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1096/2191] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:22 2022] DELETE /identity/v3/users/4dcc57d763a94d0b8ca3893fcbc7ef4b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:22.832721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:22.834391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:22.853208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb1164-aa01-4ff8-93dd-a598245e6453 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:22.937987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:22.939796 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:22.940156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:22.940510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] PATH_INFO: `/v3/projects/d9812876781a417fba6f0cdfc8228125` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:22.942281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9812876781a417fba6f0cdfc8228125)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:22.944436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:23.012580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9812876781a417fba6f0cdfc8228125', 'request_id': 'req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.022102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9812876781a417fba6f0cdfc8228125', 'request_id': 'req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.031887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9812876781a417fba6f0cdfc8228125', 'request_id': 'req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.042029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:23.047292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] The token cache is being invalidate because project d9812876781a417fba6f0cdfc8228125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:23.047838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.048793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.054393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1096/2192] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:22 2022] DELETE /identity/v3/projects/d9812876781a417fba6f0cdfc8228125 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:23.059959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:23.062365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:23.076425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0464b-a8ba-4217-82a1-5766f0fa5f0a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:23.145370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:23.146268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:23.146849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:23.147129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] PATH_INFO: `/v3/users/abff2b523dd04df0b5d489ddf22db612` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:23.148730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abff2b523dd04df0b5d489ddf22db612)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:23.150747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:23.196741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:23.204146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abff2b523dd04df0b5d489ddf22db612'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.217453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abff2b523dd04df0b5d489ddf22db612'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.224840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abff2b523dd04df0b5d489ddf22db612'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.232185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abff2b523dd04df0b5d489ddf22db612'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.232856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-738c3a28-d47c-4999-99a5-3477b6689503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abff2b523dd04df0b5d489ddf22db612'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.238906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1097/2193] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:23 2022] DELETE /identity/v3/users/abff2b523dd04df0b5d489ddf22db612 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:23.525912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:23.527379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:23.540439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc1c97-2066-43eb-93ea-ed8cbeb8a487 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:23.606658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:23.607437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:23.607721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:23.607997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] PATH_INFO: `/v3/projects/ee481d2bafdb4326af1065be59d9ce03` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:23.609232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee481d2bafdb4326af1065be59d9ce03)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:23.611137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:23.659699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee481d2bafdb4326af1065be59d9ce03', 'request_id': 'req-cd23b022-6afd-4984-aece-5218b7e2ae0d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.665886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee481d2bafdb4326af1065be59d9ce03', 'request_id': 'req-cd23b022-6afd-4984-aece-5218b7e2ae0d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.674055 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee481d2bafdb4326af1065be59d9ce03', 'request_id': 'req-cd23b022-6afd-4984-aece-5218b7e2ae0d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.685880 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] The token cache is being invalidate because project ee481d2bafdb4326af1065be59d9ce03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:23.686339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.687021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cd23b022-6afd-4984-aece-5218b7e2ae0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:23.688582 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1097/2194] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:23 2022] DELETE /identity/v3/projects/ee481d2bafdb4326af1065be59d9ce03 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:23.693691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d8ea0-74af-458a-a50d-87b5574c0b22 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:23.694114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d8ea0-74af-458a-a50d-87b5574c0b22 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:23.694501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d8ea0-74af-458a-a50d-87b5574c0b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:23.744117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f85d8ea0-74af-458a-a50d-87b5574c0b22 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:23.745462 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f85d8ea0-74af-458a-a50d-87b5574c0b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:23.813687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1098/2195] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:23.821694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38461f7-a159-48ae-be6c-3d08d9818494 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:23.821982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38461f7-a159-48ae-be6c-3d08d9818494 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:23.822257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38461f7-a159-48ae-be6c-3d08d9818494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:23.841389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f38461f7-a159-48ae-be6c-3d08d9818494 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:23.842599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f38461f7-a159-48ae-be6c-3d08d9818494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:23.850379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1098/2196] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:23.854623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d8ea0-74af-458a-a50d-87b5574c0b22 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:23.865010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d8ea0-74af-458a-a50d-87b5574c0b22 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:23.869481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd251f-3ede-4503-8c6e-9cd2e628a378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:23.870131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd251f-3ede-4503-8c6e-9cd2e628a378 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:23.870417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd251f-3ede-4503-8c6e-9cd2e628a378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:23.870698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd251f-3ede-4503-8c6e-9cd2e628a378 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:23.871429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd251f-3ede-4503-8c6e-9cd2e628a378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:05:23.871740 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd251f-3ede-4503-8c6e-9cd2e628a378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:23.873371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd251f-3ede-4503-8c6e-9cd2e628a378 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:23.884818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1099/2197] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:05:23.889749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38461f7-a159-48ae-be6c-3d08d9818494 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:23.898958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38461f7-a159-48ae-be6c-3d08d9818494 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:23.904175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f3cd-1c75-4a47-9183-687b7ba72a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:23.904816 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f3cd-1c75-4a47-9183-687b7ba72a7c admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:23.905117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f3cd-1c75-4a47-9183-687b7ba72a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:23.905409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f3cd-1c75-4a47-9183-687b7ba72a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:23.906100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684f3cd-1c75-4a47-9183-687b7ba72a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:23.908021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684f3cd-1c75-4a47-9183-687b7ba72a7c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:23.933383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1099/2198] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:23 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:23.937546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd251f-3ede-4503-8c6e-9cd2e628a378 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:23.948261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd251f-3ede-4503-8c6e-9cd2e628a378 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:24.003880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc3c18-9e47-4d99-91eb-50fb260c063f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:24.004543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc3c18-9e47-4d99-91eb-50fb260c063f admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:24.004827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc3c18-9e47-4d99-91eb-50fb260c063f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:24.005122 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc3c18-9e47-4d99-91eb-50fb260c063f admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:24.005814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc3c18-9e47-4d99-91eb-50fb260c063f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:24.007723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc3c18-9e47-4d99-91eb-50fb260c063f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:24.028010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1100/2199] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:23 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:24.032755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f3cd-1c75-4a47-9183-687b7ba72a7c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:24.043208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f3cd-1c75-4a47-9183-687b7ba72a7c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:24.048189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e59bc-e8a6-4925-bddb-60b5b4d37352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:24.048847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e59bc-e8a6-4925-bddb-60b5b4d37352 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:24.049168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e59bc-e8a6-4925-bddb-60b5b4d37352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:24.049454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e59bc-e8a6-4925-bddb-60b5b4d37352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:24.050191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e59bc-e8a6-4925-bddb-60b5b4d37352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:24.051794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e59bc-e8a6-4925-bddb-60b5b4d37352 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:24.062374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1100/2200] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:24.066579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc3c18-9e47-4d99-91eb-50fb260c063f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:24.077154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc3c18-9e47-4d99-91eb-50fb260c063f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:24.081780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9bb32-bcfa-4cde-80a0-6588cd93520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:24.082432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9bb32-bcfa-4cde-80a0-6588cd93520a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:24.082719 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9bb32-bcfa-4cde-80a0-6588cd93520a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:24.082999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9bb32-bcfa-4cde-80a0-6588cd93520a admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:24.083725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9bb32-bcfa-4cde-80a0-6588cd93520a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:24.085348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9bb32-bcfa-4cde-80a0-6588cd93520a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:24.090625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:24.098269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1101/2201] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:24.102158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e59bc-e8a6-4925-bddb-60b5b4d37352 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:24.112356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e59bc-e8a6-4925-bddb-60b5b4d37352 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:24.117012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193d669-249e-4dda-9889-eec1344923fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:24.117860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d669-249e-4dda-9889-eec1344923fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:24.118141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d669-249e-4dda-9889-eec1344923fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:24.118416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d669-249e-4dda-9889-eec1344923fd admin admin] PATH_INFO: `/v3/projects/1cbe7203160648d0a1a0a65e7f65fdf5/users/51a0f0fe2e5643b88526a1dba45b8eb6/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:24.133440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193d669-249e-4dda-9889-eec1344923fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cbe7203160648d0a1a0a65e7f65fdf5, user_id=51a0f0fe2e5643b88526a1dba45b8eb6, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:24.136227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193d669-249e-4dda-9889-eec1344923fd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:24.152123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1101/2202] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:24 2022] PUT /identity/v3/projects/1cbe7203160648d0a1a0a65e7f65fdf5/users/51a0f0fe2e5643b88526a1dba45b8eb6/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:24.163151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac9f3b-50db-4b23-9d8f-1632d868428a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:24.163463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac9f3b-50db-4b23-9d8f-1632d868428a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:24.163734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac9f3b-50db-4b23-9d8f-1632d868428a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:24.193633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a5ac9f3b-50db-4b23-9d8f-1632d868428a None None] MFA Rules not processed for user `51a0f0fe2e5643b88526a1dba45b8eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:24.195135 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a5ac9f3b-50db-4b23-9d8f-1632d868428a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:24.271460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1102/2203] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:24.282817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193d669-249e-4dda-9889-eec1344923fd admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:24.284544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7193d669-249e-4dda-9889-eec1344923fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:24.316116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:24.355438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193d669-249e-4dda-9889-eec1344923fd admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:24.359341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f51c71-730a-4f02-9699-b9cf2e3cc86b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:24.360008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f51c71-730a-4f02-9699-b9cf2e3cc86b service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:24.360293 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f51c71-730a-4f02-9699-b9cf2e3cc86b service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:24.360570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f51c71-730a-4f02-9699-b9cf2e3cc86b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:24.370053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f51c71-730a-4f02-9699-b9cf2e3cc86b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:24.371429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f51c71-730a-4f02-9699-b9cf2e3cc86b service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:24.382243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1102/2204] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:31.499718 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac9f3b-50db-4b23-9d8f-1632d868428a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:31.501236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a5ac9f3b-50db-4b23-9d8f-1632d868428a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:31.584730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac9f3b-50db-4b23-9d8f-1632d868428a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:31.590150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:31.591050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:31.591364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:31.591645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] PATH_INFO: `/v3/users/398df5c44f264a5c800f9a06a42d8aa0` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:31.592935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=398df5c44f264a5c800f9a06a42d8aa0)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:31.595499 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:31.656517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398df5c44f264a5c800f9a06a42d8aa0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:31.675158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398df5c44f264a5c800f9a06a42d8aa0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:31.683960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398df5c44f264a5c800f9a06a42d8aa0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:31.693914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398df5c44f264a5c800f9a06a42d8aa0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:31.694652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398df5c44f264a5c800f9a06a42d8aa0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:31.701440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1103/2205] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:31 2022] DELETE /identity/v3/users/398df5c44f264a5c800f9a06a42d8aa0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:31.706077 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a1440-e8ca-41e0-87ce-d9ffb9259090 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:31.706757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a1440-e8ca-41e0-87ce-d9ffb9259090 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:31.706757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a1440-e8ca-41e0-87ce-d9ffb9259090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:31.725443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b28a1440-e8ca-41e0-87ce-d9ffb9259090 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:31.726741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b28a1440-e8ca-41e0-87ce-d9ffb9259090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:31.802838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1103/2206] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:31.813639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:31.815081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:31.830424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:31.886577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e03e61-ee47-4555-9359-df0fbb01c3b8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:31.891036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f9fce-db6b-48e0-8ea7-84ec39244c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:31.891698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f9fce-db6b-48e0-8ea7-84ec39244c03 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:31.891981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f9fce-db6b-48e0-8ea7-84ec39244c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:31.892257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f9fce-db6b-48e0-8ea7-84ec39244c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:31.902621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f9fce-db6b-48e0-8ea7-84ec39244c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:31.904216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f9fce-db6b-48e0-8ea7-84ec39244c03 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:31.914531 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1104/2207] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:32.259438 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a1440-e8ca-41e0-87ce-d9ffb9259090 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:32.261147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b28a1440-e8ca-41e0-87ce-d9ffb9259090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:32.294391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a1440-e8ca-41e0-87ce-d9ffb9259090 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:32.302833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:32.304655 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:32.305497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:32.306204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] PATH_INFO: `/v3/projects/25ba6da001894ab9aa80068e985e6138` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:32.309898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ba6da001894ab9aa80068e985e6138)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:32.313216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:32.407908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ba6da001894ab9aa80068e985e6138', 'request_id': 'req-355e3866-af80-47ad-aa4f-02b37ffdff58'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.435747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ba6da001894ab9aa80068e985e6138', 'request_id': 'req-355e3866-af80-47ad-aa4f-02b37ffdff58'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.452976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ba6da001894ab9aa80068e985e6138', 'request_id': 'req-355e3866-af80-47ad-aa4f-02b37ffdff58'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.473342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] The token cache is being invalidate because project 25ba6da001894ab9aa80068e985e6138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:32.473686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.474447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.477174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1104/2208] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:32 2022] DELETE /identity/v3/projects/25ba6da001894ab9aa80068e985e6138 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:32.484337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f9fce-db6b-48e0-8ea7-84ec39244c03 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:32.497804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f9fce-db6b-48e0-8ea7-84ec39244c03 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:32.569665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:32.570506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:32.570867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:32.571152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] PATH_INFO: `/v3/users/27d64ef09d8949579112bff7e5a9615f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:32.572623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d64ef09d8949579112bff7e5a9615f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:32.574839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:32.670484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d64ef09d8949579112bff7e5a9615f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.691564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d64ef09d8949579112bff7e5a9615f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.701760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d64ef09d8949579112bff7e5a9615f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.711156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d64ef09d8949579112bff7e5a9615f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.712008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d64ef09d8949579112bff7e5a9615f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:32.720430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1105/2209] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:32 2022] DELETE /identity/v3/users/27d64ef09d8949579112bff7e5a9615f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:33.078460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:33.080220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:33.097518 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e3866-af80-47ad-aa4f-02b37ffdff58 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:33.142189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:33.143743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:33.157280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:33.174235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:33.175469 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.175820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.176173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] PATH_INFO: `/v3/projects/dc62b1fce1cc4a488d9e440c1ba5f622` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.178185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc62b1fce1cc4a488d9e440c1ba5f622)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:33.180715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:33.197457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:33.241379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b289da-d4fc-44fb-a765-0b0e47e2b879 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:33.253906 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc62b1fce1cc4a488d9e440c1ba5f622', 'request_id': 'req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.261234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc62b1fce1cc4a488d9e440c1ba5f622', 'request_id': 'req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.271786 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc62b1fce1cc4a488d9e440c1ba5f622', 'request_id': 'req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.287080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] The token cache is being invalidate because project dc62b1fce1cc4a488d9e440c1ba5f622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:33.287311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.287970 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0eb8b27-0ba7-49c0-b05e-94b50398f21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.289633 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1105/2210] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:33 2022] DELETE /identity/v3/projects/dc62b1fce1cc4a488d9e440c1ba5f622 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:33.296923 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1363b-0e05-40b9-942a-ada49a9b3211 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.297389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1363b-0e05-40b9-942a-ada49a9b3211 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.297748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1363b-0e05-40b9-942a-ada49a9b3211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.321029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bae1363b-0e05-40b9-942a-ada49a9b3211 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:33.322638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bae1363b-0e05-40b9-942a-ada49a9b3211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:33.324007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:33.325855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.326146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.326434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] PATH_INFO: `/v3/users/08be6b83eec04f058d4c37b8f55a9fce` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.328091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] RBAC: Authorizing `identity:delete_user(user_id=08be6b83eec04f058d4c37b8f55a9fce)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:33.331007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:33.339139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1106/2211] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:05:33.343455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cc5f-d2c9-44ac-94d5-464fffca1140 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.344353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cc5f-d2c9-44ac-94d5-464fffca1140 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.344732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cc5f-d2c9-44ac-94d5-464fffca1140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.373295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6021cc5f-d2c9-44ac-94d5-464fffca1140 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:33.374889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6021cc5f-d2c9-44ac-94d5-464fffca1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:33.386336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1107/2212] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:33.391812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021cc5f-d2c9-44ac-94d5-464fffca1140 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:33.402554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021cc5f-d2c9-44ac-94d5-464fffca1140 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:33.404512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08be6b83eec04f058d4c37b8f55a9fce'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.409131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ea6aa-d15c-40be-b9cf-e421daac39a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:33.410008 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ea6aa-d15c-40be-b9cf-e421daac39a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.410366 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ea6aa-d15c-40be-b9cf-e421daac39a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.410748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ea6aa-d15c-40be-b9cf-e421daac39a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.411576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ea6aa-d15c-40be-b9cf-e421daac39a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:05:33.413192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ea6aa-d15c-40be-b9cf-e421daac39a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:33.414985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ea6aa-d15c-40be-b9cf-e421daac39a9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:33.422231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08be6b83eec04f058d4c37b8f55a9fce'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.432095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08be6b83eec04f058d4c37b8f55a9fce'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.437245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1108/2213] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:05:33.438383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ea6aa-d15c-40be-b9cf-e421daac39a9 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:33.446294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08be6b83eec04f058d4c37b8f55a9fce'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.447249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2e138e9b-9a79-4e31-a7b5-3beb2cac986b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08be6b83eec04f058d4c37b8f55a9fce'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:33.449340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ea6aa-d15c-40be-b9cf-e421daac39a9 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:33.454974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1106/2214] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:33 2022] DELETE /identity/v3/users/08be6b83eec04f058d4c37b8f55a9fce => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:33.456636 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8c1b7-8ded-41b1-8c4e-27e5445f06fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:33.457337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8c1b7-8ded-41b1-8c4e-27e5445f06fe admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.457851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8c1b7-8ded-41b1-8c4e-27e5445f06fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.457962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8c1b7-8ded-41b1-8c4e-27e5445f06fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.458710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8c1b7-8ded-41b1-8c4e-27e5445f06fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:33.459495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb78448-0a71-4e0a-a14f-7d5aa1c77667 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.459907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb78448-0a71-4e0a-a14f-7d5aa1c77667 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.460278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb78448-0a71-4e0a-a14f-7d5aa1c77667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.462122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8c1b7-8ded-41b1-8c4e-27e5445f06fe admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:33.481878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0fb78448-0a71-4e0a-a14f-7d5aa1c77667 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:33.483173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0fb78448-0a71-4e0a-a14f-7d5aa1c77667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:33.501733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1109/2215] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:33 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:33.502980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8c1b7-8ded-41b1-8c4e-27e5445f06fe admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:33.506024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-55b8c1b7-8ded-41b1-8c4e-27e5445f06fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:33.554714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:33.571175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1107/2216] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:05:33.582487 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb78448-0a71-4e0a-a14f-7d5aa1c77667 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:33.584069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0fb78448-0a71-4e0a-a14f-7d5aa1c77667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:33.599326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8c1b7-8ded-41b1-8c4e-27e5445f06fe admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:33.605010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953aa09c-4b04-4bf7-991d-c5ccc9cd6844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:33.609502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953aa09c-4b04-4bf7-991d-c5ccc9cd6844 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.611452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953aa09c-4b04-4bf7-991d-c5ccc9cd6844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.613174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953aa09c-4b04-4bf7-991d-c5ccc9cd6844 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.613920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953aa09c-4b04-4bf7-991d-c5ccc9cd6844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:33.617905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953aa09c-4b04-4bf7-991d-c5ccc9cd6844 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:33.659472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1110/2217] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:33 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:05:33.664220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953aa09c-4b04-4bf7-991d-c5ccc9cd6844 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:33.666519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb78448-0a71-4e0a-a14f-7d5aa1c77667 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:33.678531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953aa09c-4b04-4bf7-991d-c5ccc9cd6844 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:33.686638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73c721-3356-4a74-9888-c90949002fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:33.687398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73c721-3356-4a74-9888-c90949002fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.687755 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73c721-3356-4a74-9888-c90949002fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.688104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73c721-3356-4a74-9888-c90949002fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.688979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73c721-3356-4a74-9888-c90949002fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:33.692104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73c721-3356-4a74-9888-c90949002fd5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:33.708931 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1111/2218] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:33.713154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73c721-3356-4a74-9888-c90949002fd5 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:33.738102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a85a0-cfc4-4834-9506-d9bc315cac43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:33.738855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a85a0-cfc4-4834-9506-d9bc315cac43 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.739448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a85a0-cfc4-4834-9506-d9bc315cac43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.739929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a85a0-cfc4-4834-9506-d9bc315cac43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.757803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a85a0-cfc4-4834-9506-d9bc315cac43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:33.758014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73c721-3356-4a74-9888-c90949002fd5 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:33.758139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a85a0-cfc4-4834-9506-d9bc315cac43 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:33.763440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedb191-b7a4-4583-b510-074f07ba751f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:33.764362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedb191-b7a4-4583-b510-074f07ba751f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.764716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedb191-b7a4-4583-b510-074f07ba751f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.765235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedb191-b7a4-4583-b510-074f07ba751f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.766051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedb191-b7a4-4583-b510-074f07ba751f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:33.767737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedb191-b7a4-4583-b510-074f07ba751f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:33.776000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:33.781370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1108/2219] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:05:33.784456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1112/2220] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:33.785131 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a85a0-cfc4-4834-9506-d9bc315cac43 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:33.799631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a85a0-cfc4-4834-9506-d9bc315cac43 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:33.805023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7606a-1b2c-405c-af54-20aca658b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:33.805953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7606a-1b2c-405c-af54-20aca658b5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.806650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7606a-1b2c-405c-af54-20aca658b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.806650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7606a-1b2c-405c-af54-20aca658b5ec admin admin] PATH_INFO: `/v3/projects/935d98b5829a44e0b7da107a1e29272d/users/c1edc182b3384545bfd4ca0358e8bf40/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.828744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7606a-1b2c-405c-af54-20aca658b5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=935d98b5829a44e0b7da107a1e29272d, user_id=c1edc182b3384545bfd4ca0358e8bf40, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:33.835433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7606a-1b2c-405c-af54-20aca658b5ec admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:33.859663 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1109/2221] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:33 2022] PUT /identity/v3/projects/935d98b5829a44e0b7da107a1e29272d/users/c1edc182b3384545bfd4ca0358e8bf40/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:33.874905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fa36b-1cfc-4b09-a0ce-853abe957e73 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:33.875360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fa36b-1cfc-4b09-a0ce-853abe957e73 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:33.875753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fa36b-1cfc-4b09-a0ce-853abe957e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:33.911700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1b3fa36b-1cfc-4b09-a0ce-853abe957e73 None None] MFA Rules not processed for user `c1edc182b3384545bfd4ca0358e8bf40`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:33.913351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1b3fa36b-1cfc-4b09-a0ce-853abe957e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:34.110049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1113/2222] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:34.123389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7606a-1b2c-405c-af54-20aca658b5ec admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:34.125302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3cc7606a-1b2c-405c-af54-20aca658b5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:34.197479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fa36b-1cfc-4b09-a0ce-853abe957e73 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:34.199285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1b3fa36b-1cfc-4b09-a0ce-853abe957e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:34.221521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7606a-1b2c-405c-af54-20aca658b5ec admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:34.226456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c71dd3-3dc6-4a15-ab54-65a1d9424d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:34.227143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c71dd3-3dc6-4a15-ab54-65a1d9424d14 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.227434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c71dd3-3dc6-4a15-ab54-65a1d9424d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.227717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c71dd3-3dc6-4a15-ab54-65a1d9424d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.241003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c71dd3-3dc6-4a15-ab54-65a1d9424d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:34.242776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c71dd3-3dc6-4a15-ab54-65a1d9424d14 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:34.254554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1110/2223] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:34.328274 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fa36b-1cfc-4b09-a0ce-853abe957e73 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:34.336416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:34.337380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.337664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.338374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] PATH_INFO: `/v3/projects/e4d9e7ae30ef42e189a3cf86e676b50d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.339242 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d9e7ae30ef42e189a3cf86e676b50d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:34.341262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:34.406423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d9e7ae30ef42e189a3cf86e676b50d', 'request_id': 'req-158767ef-644e-4822-9d41-0878d7ff2490'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:34.415643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d9e7ae30ef42e189a3cf86e676b50d', 'request_id': 'req-158767ef-644e-4822-9d41-0878d7ff2490'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:34.426652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d9e7ae30ef42e189a3cf86e676b50d', 'request_id': 'req-158767ef-644e-4822-9d41-0878d7ff2490'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:34.442080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] The token cache is being invalidate because project e4d9e7ae30ef42e189a3cf86e676b50d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:34.442555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:34.443409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-158767ef-644e-4822-9d41-0878d7ff2490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:34.449332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1114/2224] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:34 2022] DELETE /identity/v3/projects/e4d9e7ae30ef42e189a3cf86e676b50d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:34.451370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ed060-2b0f-4502-97da-5ca021b521ff None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.451730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ed060-2b0f-4502-97da-5ca021b521ff None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.452016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ed060-2b0f-4502-97da-5ca021b521ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.476306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d97ed060-2b0f-4502-97da-5ca021b521ff None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:34.477923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d97ed060-2b0f-4502-97da-5ca021b521ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:34.480746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42882668-5b0b-468c-ad01-f47c7aa4da64 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.481853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42882668-5b0b-468c-ad01-f47c7aa4da64 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.482302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42882668-5b0b-468c-ad01-f47c7aa4da64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.503539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-42882668-5b0b-468c-ad01-f47c7aa4da64 None None] MFA Rules not processed for user `eeafe22d22f648d6aefcc6a308d4fc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:34.505272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-42882668-5b0b-468c-ad01-f47c7aa4da64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:34.514607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:34.523266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:34.571190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1111/2225] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:34.579152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949e7ed-4c54-46b6-a7b5-7eac02353e3f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.579337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949e7ed-4c54-46b6-a7b5-7eac02353e3f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.579777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949e7ed-4c54-46b6-a7b5-7eac02353e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.596157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1115/2226] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:34.611677 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5949e7ed-4c54-46b6-a7b5-7eac02353e3f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:34.613233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5949e7ed-4c54-46b6-a7b5-7eac02353e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:34.629700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1112/2227] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:05:34.631875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42882668-5b0b-468c-ad01-f47c7aa4da64 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:34.643702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42882668-5b0b-468c-ad01-f47c7aa4da64 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:34.649204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e904908-4588-49f2-893d-4ea02b7cd81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:34.649720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e904908-4588-49f2-893d-4ea02b7cd81b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.650011 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e904908-4588-49f2-893d-4ea02b7cd81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.650289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e904908-4588-49f2-893d-4ea02b7cd81b admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.651064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e904908-4588-49f2-893d-4ea02b7cd81b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:05:34.651464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e904908-4588-49f2-893d-4ea02b7cd81b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:34.653321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e904908-4588-49f2-893d-4ea02b7cd81b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:34.671437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1116/2228] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:05:34.675857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949e7ed-4c54-46b6-a7b5-7eac02353e3f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:34.686033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949e7ed-4c54-46b6-a7b5-7eac02353e3f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:34.692012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db56db0-1e48-4d3a-a87d-bd34790acb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:34.692803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db56db0-1e48-4d3a-a87d-bd34790acb0a admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.693043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db56db0-1e48-4d3a-a87d-bd34790acb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.693392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db56db0-1e48-4d3a-a87d-bd34790acb0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.694233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db56db0-1e48-4d3a-a87d-bd34790acb0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:34.696326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db56db0-1e48-4d3a-a87d-bd34790acb0a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:34.724636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1113/2229] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:34 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:34.729040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e904908-4588-49f2-893d-4ea02b7cd81b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:34.740000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e904908-4588-49f2-893d-4ea02b7cd81b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:34.804271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7e231-a540-451e-9343-fc22c7f66a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:34.805029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7e231-a540-451e-9343-fc22c7f66a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.805447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7e231-a540-451e-9343-fc22c7f66a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.805789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7e231-a540-451e-9343-fc22c7f66a41 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.806585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7e231-a540-451e-9343-fc22c7f66a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:34.808828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7e231-a540-451e-9343-fc22c7f66a41 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:34.834533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1117/2230] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:34 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:34.836028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db56db0-1e48-4d3a-a87d-bd34790acb0a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:34.847577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db56db0-1e48-4d3a-a87d-bd34790acb0a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:34.852416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa3613-da29-4fb2-9f8c-62eaa38ca130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:34.853104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3613-da29-4fb2-9f8c-62eaa38ca130 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.854041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3613-da29-4fb2-9f8c-62eaa38ca130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.854041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3613-da29-4fb2-9f8c-62eaa38ca130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.854406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa3613-da29-4fb2-9f8c-62eaa38ca130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:34.856059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa3613-da29-4fb2-9f8c-62eaa38ca130 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:34.876917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1114/2231] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:34.880725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7e231-a540-451e-9343-fc22c7f66a41 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:34.894994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7e231-a540-451e-9343-fc22c7f66a41 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:34.900651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf54db-3007-491e-b8ad-4ce9bc081ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:34.901367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf54db-3007-491e-b8ad-4ce9bc081ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.901651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf54db-3007-491e-b8ad-4ce9bc081ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.901929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf54db-3007-491e-b8ad-4ce9bc081ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.902667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf54db-3007-491e-b8ad-4ce9bc081ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:34.904928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf54db-3007-491e-b8ad-4ce9bc081ab9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:34.919753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1118/2232] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:34.923831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa3613-da29-4fb2-9f8c-62eaa38ca130 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:34.934354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa3613-da29-4fb2-9f8c-62eaa38ca130 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:34.939196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d0cbc-784c-459c-a933-4a91e6e11f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:34.940082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d0cbc-784c-459c-a933-4a91e6e11f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.940363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d0cbc-784c-459c-a933-4a91e6e11f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.940906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d0cbc-784c-459c-a933-4a91e6e11f53 admin admin] PATH_INFO: `/v3/projects/4202625ef50b44de9a96563feb874a5a/users/c19aa025ff3046229cf0b8041058285e/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:34.957998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d0cbc-784c-459c-a933-4a91e6e11f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4202625ef50b44de9a96563feb874a5a, user_id=c19aa025ff3046229cf0b8041058285e, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:34.960800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d0cbc-784c-459c-a933-4a91e6e11f53 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:34.977864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1115/2233] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:34 2022] PUT /identity/v3/projects/4202625ef50b44de9a96563feb874a5a/users/c19aa025ff3046229cf0b8041058285e/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:34.983569 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cefa7-7702-4ab3-a316-2a3dff98922b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:34.983859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cefa7-7702-4ab3-a316-2a3dff98922b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:34.984139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cefa7-7702-4ab3-a316-2a3dff98922b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:35.001898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fc8cefa7-7702-4ab3-a316-2a3dff98922b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:35.003288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fc8cefa7-7702-4ab3-a316-2a3dff98922b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:35.069329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1119/2234] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:35.085048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d0cbc-784c-459c-a933-4a91e6e11f53 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:35.086819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a45d0cbc-784c-459c-a933-4a91e6e11f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:35.174047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d0cbc-784c-459c-a933-4a91e6e11f53 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:35.178250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a79a9-6840-4d22-930e-24be2f0e7d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:35.179114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a79a9-6840-4d22-930e-24be2f0e7d31 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:35.179403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a79a9-6840-4d22-930e-24be2f0e7d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:35.180437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a79a9-6840-4d22-930e-24be2f0e7d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:35.193554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a79a9-6840-4d22-930e-24be2f0e7d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:35.195154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a79a9-6840-4d22-930e-24be2f0e7d31 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:35.206558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1116/2235] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:39.553247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fa0c2-f73a-4a24-b1a0-d8489f742756 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:39.553784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fa0c2-f73a-4a24-b1a0-d8489f742756 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:39.553884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fa0c2-f73a-4a24-b1a0-d8489f742756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:39.585138 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6c7fa0c2-f73a-4a24-b1a0-d8489f742756 None None] MFA Rules not processed for user `c19aa025ff3046229cf0b8041058285e`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:39.586379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6c7fa0c2-f73a-4a24-b1a0-d8489f742756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:39.606906 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:39.663239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1120/2236] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:39.664394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a79a9-6840-4d22-930e-24be2f0e7d31 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:39.676608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a79a9-6840-4d22-930e-24be2f0e7d31 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:39.681910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7aab47-201c-4b86-8ecf-4f65fc4bbb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:39.682696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aab47-201c-4b86-8ecf-4f65fc4bbb95 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:39.683087 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aab47-201c-4b86-8ecf-4f65fc4bbb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:39.683585 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aab47-201c-4b86-8ecf-4f65fc4bbb95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:39.685308 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7aab47-201c-4b86-8ecf-4f65fc4bbb95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:39.687495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7aab47-201c-4b86-8ecf-4f65fc4bbb95 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:39.722583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1117/2237] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:39 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:39.723828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fa0c2-f73a-4a24-b1a0-d8489f742756 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:39.736513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fa0c2-f73a-4a24-b1a0-d8489f742756 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:39.804311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a98c8b-a9d8-4fbf-80f5-5934f9c3d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:39.805191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a98c8b-a9d8-4fbf-80f5-5934f9c3d364 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:39.805549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a98c8b-a9d8-4fbf-80f5-5934f9c3d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:39.805833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a98c8b-a9d8-4fbf-80f5-5934f9c3d364 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:39.806546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a98c8b-a9d8-4fbf-80f5-5934f9c3d364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:39.808501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a98c8b-a9d8-4fbf-80f5-5934f9c3d364 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:39.836361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1121/2238] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:39 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:39.837768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7aab47-201c-4b86-8ecf-4f65fc4bbb95 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:39.848477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7aab47-201c-4b86-8ecf-4f65fc4bbb95 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:39.853423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c9704-b9ab-4825-b4da-5ee7c1471873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:39.854158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c9704-b9ab-4825-b4da-5ee7c1471873 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:39.854514 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c9704-b9ab-4825-b4da-5ee7c1471873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:39.854866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c9704-b9ab-4825-b4da-5ee7c1471873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:39.855669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c9704-b9ab-4825-b4da-5ee7c1471873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:39.857925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c9704-b9ab-4825-b4da-5ee7c1471873 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:39.869737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1118/2239] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:39.874377 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a98c8b-a9d8-4fbf-80f5-5934f9c3d364 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:39.886061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a98c8b-a9d8-4fbf-80f5-5934f9c3d364 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:39.891079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985e297-546e-4515-834f-fa448591f9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:39.891984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985e297-546e-4515-834f-fa448591f9cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:39.892358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985e297-546e-4515-834f-fa448591f9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:39.892839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985e297-546e-4515-834f-fa448591f9cf admin admin] PATH_INFO: `/v3/projects/5f61dcd16eac4e22bb135b168d015d4d/users/4468571e5cb1492eb10c7cd61e0b6a19/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:39.911077 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985e297-546e-4515-834f-fa448591f9cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f61dcd16eac4e22bb135b168d015d4d, user_id=4468571e5cb1492eb10c7cd61e0b6a19, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:39.913792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985e297-546e-4515-834f-fa448591f9cf admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:39.932652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1122/2240] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:39 2022] PUT /identity/v3/projects/5f61dcd16eac4e22bb135b168d015d4d/users/4468571e5cb1492eb10c7cd61e0b6a19/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:39.937393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c9704-b9ab-4825-b4da-5ee7c1471873 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:39.949263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c9704-b9ab-4825-b4da-5ee7c1471873 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:40.012937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:40.036103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880129fb-6862-40e9-af34-e446bea36629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:40.036897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880129fb-6862-40e9-af34-e446bea36629 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:40.037287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880129fb-6862-40e9-af34-e446bea36629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:40.037648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880129fb-6862-40e9-af34-e446bea36629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:40.038711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880129fb-6862-40e9-af34-e446bea36629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:40.041508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880129fb-6862-40e9-af34-e446bea36629 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:40.067852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1119/2241] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:40.072716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985e297-546e-4515-834f-fa448591f9cf admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:40.090070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985e297-546e-4515-834f-fa448591f9cf admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:40.107806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365181c-8e4e-4a12-98df-4fb4bd9d5878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:40.109121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365181c-8e4e-4a12-98df-4fb4bd9d5878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:40.109246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365181c-8e4e-4a12-98df-4fb4bd9d5878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:40.110066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365181c-8e4e-4a12-98df-4fb4bd9d5878 admin admin] PATH_INFO: `/v3/projects/5f61dcd16eac4e22bb135b168d015d4d/users/4468571e5cb1492eb10c7cd61e0b6a19/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:40.114659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365181c-8e4e-4a12-98df-4fb4bd9d5878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f61dcd16eac4e22bb135b168d015d4d, user_id=4468571e5cb1492eb10c7cd61e0b6a19, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:40.138726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365181c-8e4e-4a12-98df-4fb4bd9d5878 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:40.162904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1123/2242] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:40 2022] PUT /identity/v3/projects/5f61dcd16eac4e22bb135b168d015d4d/users/4468571e5cb1492eb10c7cd61e0b6a19/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:46.111067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0da32-f6cb-4331-98ce-40137f7f9230 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:46.112470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0da32-f6cb-4331-98ce-40137f7f9230 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:46.112938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0da32-f6cb-4331-98ce-40137f7f9230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:46.170274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f3d0da32-f6cb-4331-98ce-40137f7f9230 None None] MFA Rules not processed for user `4468571e5cb1492eb10c7cd61e0b6a19`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:46.173287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f3d0da32-f6cb-4331-98ce-40137f7f9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:46.304111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1120/2243] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:46.323280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365181c-8e4e-4a12-98df-4fb4bd9d5878 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:46.331650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c365181c-8e4e-4a12-98df-4fb4bd9d5878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:46.442440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:46.483286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365181c-8e4e-4a12-98df-4fb4bd9d5878 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:46.490223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f8369-d06e-4f03-b9cb-b3e1cfb7f1cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:46.491015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f8369-d06e-4f03-b9cb-b3e1cfb7f1cd service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:46.491367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f8369-d06e-4f03-b9cb-b3e1cfb7f1cd service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:46.491779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f8369-d06e-4f03-b9cb-b3e1cfb7f1cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:46.576721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f8369-d06e-4f03-b9cb-b3e1cfb7f1cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:46.591996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f8369-d06e-4f03-b9cb-b3e1cfb7f1cd service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:46.616756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1124/2244] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:47.188627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0da32-f6cb-4331-98ce-40137f7f9230 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:47.192227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f3d0da32-f6cb-4331-98ce-40137f7f9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:47.352311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0da32-f6cb-4331-98ce-40137f7f9230 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:47.359764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:47.360545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:47.360820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:47.361301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:47.364069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:47.392519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:47.396491 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:47.413842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1121/2245] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:48.650714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70a599-73d6-470a-8cd3-88f9edd91252 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:48.651254 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70a599-73d6-470a-8cd3-88f9edd91252 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:48.651254 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70a599-73d6-470a-8cd3-88f9edd91252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:48.669820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2f70a599-73d6-470a-8cd3-88f9edd91252 None None] MFA Rules not processed for user `c19aa025ff3046229cf0b8041058285e`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:48.671152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2f70a599-73d6-470a-8cd3-88f9edd91252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:48.679046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1125/2246] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:48.902470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:48.904031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:48.953697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:49.010358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718887a-17fb-4ea5-baca-18711e70c400 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:49.030115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:49.032497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:49.032843 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:49.033439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] PATH_INFO: `/v3/users/c1edc182b3384545bfd4ca0358e8bf40` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:49.037062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1edc182b3384545bfd4ca0358e8bf40)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:49.045114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:49.161975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1edc182b3384545bfd4ca0358e8bf40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.196299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1edc182b3384545bfd4ca0358e8bf40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.217743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1edc182b3384545bfd4ca0358e8bf40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.227303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1edc182b3384545bfd4ca0358e8bf40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.228208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1edc182b3384545bfd4ca0358e8bf40'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.236384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1122/2247] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:05:48 2022] DELETE /identity/v3/users/c1edc182b3384545bfd4ca0358e8bf40 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:49.240858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cb142-0065-45a5-8591-9f0ccffe3d27 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:49.241176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cb142-0065-45a5-8591-9f0ccffe3d27 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:49.241463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cb142-0065-45a5-8591-9f0ccffe3d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:49.296770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fd7cb142-0065-45a5-8591-9f0ccffe3d27 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:49.298294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fd7cb142-0065-45a5-8591-9f0ccffe3d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:49.396567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1126/2248] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:49.406818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:49.409016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:49.503675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dfccb-4721-455e-80da-2aac538436f6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:49.509256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798c826-d186-4848-ac07-06f212e44344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:49.510004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798c826-d186-4848-ac07-06f212e44344 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:49.510480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798c826-d186-4848-ac07-06f212e44344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:49.510859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798c826-d186-4848-ac07-06f212e44344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:49.521241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798c826-d186-4848-ac07-06f212e44344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:49.522692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798c826-d186-4848-ac07-06f212e44344 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:49.533013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1123/2249] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:49.834841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cb142-0065-45a5-8591-9f0ccffe3d27 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:49.837260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fd7cb142-0065-45a5-8591-9f0ccffe3d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:49.851779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cb142-0065-45a5-8591-9f0ccffe3d27 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:49.856885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:49.857982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:49.858272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:49.858559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] PATH_INFO: `/v3/projects/935d98b5829a44e0b7da107a1e29272d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:49.859857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935d98b5829a44e0b7da107a1e29272d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:49.862035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:49.918442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935d98b5829a44e0b7da107a1e29272d', 'request_id': 'req-f037aaa4-1387-4642-b66b-da852dde8b36'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.925010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935d98b5829a44e0b7da107a1e29272d', 'request_id': 'req-f037aaa4-1387-4642-b66b-da852dde8b36'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.932895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935d98b5829a44e0b7da107a1e29272d', 'request_id': 'req-f037aaa4-1387-4642-b66b-da852dde8b36'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.941163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:49.945458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] The token cache is being invalidate because project 935d98b5829a44e0b7da107a1e29272d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:05:49.945948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.946852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f037aaa4-1387-4642-b66b-da852dde8b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:05:49.948802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1127/2250] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:05:49 2022] DELETE /identity/v3/projects/935d98b5829a44e0b7da107a1e29272d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:49.955082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c16709-91ea-4427-ab54-921360c76b38 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:49.955628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c16709-91ea-4427-ab54-921360c76b38 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:49.955628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c16709-91ea-4427-ab54-921360c76b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:49.974431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e5c16709-91ea-4427-ab54-921360c76b38 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:49.975682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e5c16709-91ea-4427-ab54-921360c76b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:50.028977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:50.043775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1124/2251] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:50.052720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9effa-60e7-4078-9912-994fe39922a5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.053021 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9effa-60e7-4078-9912-994fe39922a5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.053357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9effa-60e7-4078-9912-994fe39922a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.075556 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9fc9effa-60e7-4078-9912-994fe39922a5 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:50.076874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9fc9effa-60e7-4078-9912-994fe39922a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:50.085107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1128/2252] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:50.089478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c16709-91ea-4427-ab54-921360c76b38 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.099576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c16709-91ea-4427-ab54-921360c76b38 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.109711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d27425-452a-4272-93ab-aa868bd2b96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.110203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d27425-452a-4272-93ab-aa868bd2b96b admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.110495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d27425-452a-4272-93ab-aa868bd2b96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.110876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d27425-452a-4272-93ab-aa868bd2b96b admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.111712 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d27425-452a-4272-93ab-aa868bd2b96b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:05:50.111989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d27425-452a-4272-93ab-aa868bd2b96b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.113796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d27425-452a-4272-93ab-aa868bd2b96b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.129168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1125/2253] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:05:50.133909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9effa-60e7-4078-9912-994fe39922a5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.144323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9effa-60e7-4078-9912-994fe39922a5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.149034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a468cf-8ecc-424e-aed2-c0ee42fa14c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.149753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a468cf-8ecc-424e-aed2-c0ee42fa14c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.150045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a468cf-8ecc-424e-aed2-c0ee42fa14c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.150326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a468cf-8ecc-424e-aed2-c0ee42fa14c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.151032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a468cf-8ecc-424e-aed2-c0ee42fa14c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.152984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a468cf-8ecc-424e-aed2-c0ee42fa14c1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.179381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1129/2254] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:50 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:50.184109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d27425-452a-4272-93ab-aa868bd2b96b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.194978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d27425-452a-4272-93ab-aa868bd2b96b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.256460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e9638-a7c2-4834-8552-241516194504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.257286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e9638-a7c2-4834-8552-241516194504 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.257633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e9638-a7c2-4834-8552-241516194504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.257978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e9638-a7c2-4834-8552-241516194504 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.258773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e9638-a7c2-4834-8552-241516194504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.261175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e9638-a7c2-4834-8552-241516194504 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.288732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1126/2255] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:50 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:50.293472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a468cf-8ecc-424e-aed2-c0ee42fa14c1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.304598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a468cf-8ecc-424e-aed2-c0ee42fa14c1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.310462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb166b-f669-43be-812b-604a570122ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.311134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb166b-f669-43be-812b-604a570122ad admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.311421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb166b-f669-43be-812b-604a570122ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.311705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb166b-f669-43be-812b-604a570122ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.312442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb166b-f669-43be-812b-604a570122ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.314092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb166b-f669-43be-812b-604a570122ad admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.326620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1130/2256] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:50.330176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e9638-a7c2-4834-8552-241516194504 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.343133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e9638-a7c2-4834-8552-241516194504 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.348647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfdabe-6ed4-4e1c-b1b1-de0f04078cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.349374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfdabe-6ed4-4e1c-b1b1-de0f04078cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.349667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfdabe-6ed4-4e1c-b1b1-de0f04078cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.349956 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfdabe-6ed4-4e1c-b1b1-de0f04078cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.350699 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfdabe-6ed4-4e1c-b1b1-de0f04078cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.352328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfdabe-6ed4-4e1c-b1b1-de0f04078cdc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.363726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1127/2257] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:50.367841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb166b-f669-43be-812b-604a570122ad admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.379088 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb166b-f669-43be-812b-604a570122ad admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.383447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60cc3b-02ab-4380-8cf6-c496f14e42f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.384337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60cc3b-02ab-4380-8cf6-c496f14e42f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.384651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60cc3b-02ab-4380-8cf6-c496f14e42f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.384911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60cc3b-02ab-4380-8cf6-c496f14e42f5 admin admin] PATH_INFO: `/v3/projects/6f2fe178a0034089ab01c9dda18147c7/users/1bc4a71f23254779b0209a1954c6743e/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.402066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60cc3b-02ab-4380-8cf6-c496f14e42f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2fe178a0034089ab01c9dda18147c7, user_id=1bc4a71f23254779b0209a1954c6743e, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.405640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60cc3b-02ab-4380-8cf6-c496f14e42f5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.422611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1131/2258] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:50 2022] PUT /identity/v3/projects/6f2fe178a0034089ab01c9dda18147c7/users/1bc4a71f23254779b0209a1954c6743e/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:50.432904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bc5f3-eaae-47fe-8fac-9664822c8bd7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.433308 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bc5f3-eaae-47fe-8fac-9664822c8bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.433696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bc5f3-eaae-47fe-8fac-9664822c8bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.462618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-229bc5f3-eaae-47fe-8fac-9664822c8bd7 None None] MFA Rules not processed for user `1bc4a71f23254779b0209a1954c6743e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:50.464621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-229bc5f3-eaae-47fe-8fac-9664822c8bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:50.542381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1128/2259] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:50.547085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60cc3b-02ab-4380-8cf6-c496f14e42f5 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.615506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60cc3b-02ab-4380-8cf6-c496f14e42f5 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.624722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69063758-51f9-4ab8-bb52-64f606f23f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.624722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063758-51f9-4ab8-bb52-64f606f23f5d admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.624722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063758-51f9-4ab8-bb52-64f606f23f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.624722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063758-51f9-4ab8-bb52-64f606f23f5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.625388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69063758-51f9-4ab8-bb52-64f606f23f5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.627080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69063758-51f9-4ab8-bb52-64f606f23f5d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.659364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1132/2260] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:50 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:50.660546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bc5f3-eaae-47fe-8fac-9664822c8bd7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.672037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bc5f3-eaae-47fe-8fac-9664822c8bd7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.726871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:50.739950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d3d92-d9cf-4335-a2cf-5cf490f3e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.740791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d3d92-d9cf-4335-a2cf-5cf490f3e439 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.741342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d3d92-d9cf-4335-a2cf-5cf490f3e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.741626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d3d92-d9cf-4335-a2cf-5cf490f3e439 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.742446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d3d92-d9cf-4335-a2cf-5cf490f3e439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.745364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d3d92-d9cf-4335-a2cf-5cf490f3e439 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.769942 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1129/2261] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:50 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:50.774067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69063758-51f9-4ab8-bb52-64f606f23f5d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.784845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69063758-51f9-4ab8-bb52-64f606f23f5d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.789674 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3792e1-52dc-44cf-823b-a822ed35d3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.790363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3792e1-52dc-44cf-823b-a822ed35d3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.790649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3792e1-52dc-44cf-823b-a822ed35d3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.790927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3792e1-52dc-44cf-823b-a822ed35d3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.791681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3792e1-52dc-44cf-823b-a822ed35d3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.793391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3792e1-52dc-44cf-823b-a822ed35d3d6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.805598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1133/2262] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:50.810467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d3d92-d9cf-4335-a2cf-5cf490f3e439 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.821444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d3d92-d9cf-4335-a2cf-5cf490f3e439 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.827157 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59a444-af1d-4bc4-ba74-ae1ac29e3c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.827849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59a444-af1d-4bc4-ba74-ae1ac29e3c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.828139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59a444-af1d-4bc4-ba74-ae1ac29e3c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.828423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59a444-af1d-4bc4-ba74-ae1ac29e3c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.829217 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59a444-af1d-4bc4-ba74-ae1ac29e3c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.830881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59a444-af1d-4bc4-ba74-ae1ac29e3c43 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.842325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1130/2263] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:50.846605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3792e1-52dc-44cf-823b-a822ed35d3d6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:50.857051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3792e1-52dc-44cf-823b-a822ed35d3d6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:50.861829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f2662-0509-4194-8476-9c3aa0d2a1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:50.862736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f2662-0509-4194-8476-9c3aa0d2a1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.863020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f2662-0509-4194-8476-9c3aa0d2a1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.863297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f2662-0509-4194-8476-9c3aa0d2a1a9 admin admin] PATH_INFO: `/v3/projects/1611410aab444d2a979a1e021f0f4dac/users/b172edc92a924524b0d0314f5febf69c/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.880605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f2662-0509-4194-8476-9c3aa0d2a1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1611410aab444d2a979a1e021f0f4dac, user_id=b172edc92a924524b0d0314f5febf69c, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:50.883311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f2662-0509-4194-8476-9c3aa0d2a1a9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:50.898584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1134/2264] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:50 2022] PUT /identity/v3/projects/1611410aab444d2a979a1e021f0f4dac/users/b172edc92a924524b0d0314f5febf69c/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:50.909189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15733-f891-4fb3-8e44-d1168fc50a69 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:50.909588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15733-f891-4fb3-8e44-d1168fc50a69 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:50.909869 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15733-f891-4fb3-8e44-d1168fc50a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:50.942540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-69a15733-f891-4fb3-8e44-d1168fc50a69 None None] MFA Rules not processed for user `b172edc92a924524b0d0314f5febf69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:50.943798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-69a15733-f891-4fb3-8e44-d1168fc50a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:51.010319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1131/2265] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:51.014850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f2662-0509-4194-8476-9c3aa0d2a1a9 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:51.071157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:51.089298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f2662-0509-4194-8476-9c3aa0d2a1a9 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:51.094372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d6f2c-eae8-4628-92d7-482f9fa80676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:51.094980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d6f2c-eae8-4628-92d7-482f9fa80676 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:51.095269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d6f2c-eae8-4628-92d7-482f9fa80676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:51.095550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d6f2c-eae8-4628-92d7-482f9fa80676 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:51.096409 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d6f2c-eae8-4628-92d7-482f9fa80676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:51.101449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d6f2c-eae8-4628-92d7-482f9fa80676 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:51.135408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1135/2266] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:05:51 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:51.139741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15733-f891-4fb3-8e44-d1168fc50a69 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:51.151295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15733-f891-4fb3-8e44-d1168fc50a69 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:51.217290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d75e2f-9875-44dc-8626-33318b18c72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:51.217977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d75e2f-9875-44dc-8626-33318b18c72b admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:51.218266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d75e2f-9875-44dc-8626-33318b18c72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:51.218572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d75e2f-9875-44dc-8626-33318b18c72b admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:51.219335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d75e2f-9875-44dc-8626-33318b18c72b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:51.221418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d75e2f-9875-44dc-8626-33318b18c72b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:51.248780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1132/2267] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:05:51 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:05:51.253201 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d6f2c-eae8-4628-92d7-482f9fa80676 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:51.272682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d6f2c-eae8-4628-92d7-482f9fa80676 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:51.278204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18aac9-3688-4ecf-96b2-0eacc8659872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:51.278878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18aac9-3688-4ecf-96b2-0eacc8659872 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:51.279164 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18aac9-3688-4ecf-96b2-0eacc8659872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:51.279447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18aac9-3688-4ecf-96b2-0eacc8659872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:51.280182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18aac9-3688-4ecf-96b2-0eacc8659872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:51.281855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18aac9-3688-4ecf-96b2-0eacc8659872 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:51.294394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1136/2268] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:05:51.298897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d75e2f-9875-44dc-8626-33318b18c72b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:51.309846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d75e2f-9875-44dc-8626-33318b18c72b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:51.314906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f04d52-c58a-4b95-ab1e-5aa79154ed88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:51.316102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f04d52-c58a-4b95-ab1e-5aa79154ed88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:51.316396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f04d52-c58a-4b95-ab1e-5aa79154ed88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:51.316661 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f04d52-c58a-4b95-ab1e-5aa79154ed88 admin admin] PATH_INFO: `/v3/projects/f9085219f9a24111bfff2a852b4fa419/users/55b62488f46f4948bbc3a1e0abdb2139/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:51.334790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f04d52-c58a-4b95-ab1e-5aa79154ed88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9085219f9a24111bfff2a852b4fa419, user_id=55b62488f46f4948bbc3a1e0abdb2139, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:51.337505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f04d52-c58a-4b95-ab1e-5aa79154ed88 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:51.354196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1133/2269] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:51 2022] PUT /identity/v3/projects/f9085219f9a24111bfff2a852b4fa419/users/55b62488f46f4948bbc3a1e0abdb2139/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:05:51.358811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18aac9-3688-4ecf-96b2-0eacc8659872 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:51.369504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18aac9-3688-4ecf-96b2-0eacc8659872 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:51.452389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d263c-810d-4fa7-b4b7-d95fbab4bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:51.453919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d263c-810d-4fa7-b4b7-d95fbab4bdfa admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:51.454250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d263c-810d-4fa7-b4b7-d95fbab4bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:51.454578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d263c-810d-4fa7-b4b7-d95fbab4bdfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:51.455673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d263c-810d-4fa7-b4b7-d95fbab4bdfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:51.457514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d263c-810d-4fa7-b4b7-d95fbab4bdfa admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:51.479575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1137/2270] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:05:51.484333 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f04d52-c58a-4b95-ab1e-5aa79154ed88 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:51.498870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f04d52-c58a-4b95-ab1e-5aa79154ed88 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:51.509164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662730f6-f33e-4a77-94d7-070af3dff862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:51.510466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662730f6-f33e-4a77-94d7-070af3dff862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:51.510765 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662730f6-f33e-4a77-94d7-070af3dff862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:51.511032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662730f6-f33e-4a77-94d7-070af3dff862 admin admin] PATH_INFO: `/v3/projects/f9085219f9a24111bfff2a852b4fa419/users/55b62488f46f4948bbc3a1e0abdb2139/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:51.513657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662730f6-f33e-4a77-94d7-070af3dff862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9085219f9a24111bfff2a852b4fa419, user_id=55b62488f46f4948bbc3a1e0abdb2139, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:51.516658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662730f6-f33e-4a77-94d7-070af3dff862 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:51.530408 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1134/2271] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:05:51 2022] PUT /identity/v3/projects/f9085219f9a24111bfff2a852b4fa419/users/55b62488f46f4948bbc3a1e0abdb2139/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:05:51.541243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f488e33-c04e-49ec-ab09-17dd0f94fe1e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:51.541537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f488e33-c04e-49ec-ab09-17dd0f94fe1e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:51.542216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f488e33-c04e-49ec-ab09-17dd0f94fe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:51.576587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5f488e33-c04e-49ec-ab09-17dd0f94fe1e None None] MFA Rules not processed for user `55b62488f46f4948bbc3a1e0abdb2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:05:51.577846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5f488e33-c04e-49ec-ab09-17dd0f94fe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:51.669776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1138/2272] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:05:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:05:51.680034 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662730f6-f33e-4a77-94d7-070af3dff862 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:51.683986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-662730f6-f33e-4a77-94d7-070af3dff862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:51.706618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:51.794709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662730f6-f33e-4a77-94d7-070af3dff862 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:51.798840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d71cce-5f0c-4987-9a06-58a9c81d3829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:51.799472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d71cce-5f0c-4987-9a06-58a9c81d3829 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:51.799759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d71cce-5f0c-4987-9a06-58a9c81d3829 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:51.800032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d71cce-5f0c-4987-9a06-58a9c81d3829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:51.870901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d71cce-5f0c-4987-9a06-58a9c81d3829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:51.872489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d71cce-5f0c-4987-9a06-58a9c81d3829 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:51.882443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1135/2273] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:05:54.524901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f488e33-c04e-49ec-ab09-17dd0f94fe1e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:54.536697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f488e33-c04e-49ec-ab09-17dd0f94fe1e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:54.540988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b78ea-7777-4605-8f4b-13de4e3543a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:54.541724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b78ea-7777-4605-8f4b-13de4e3543a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:54.542012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b78ea-7777-4605-8f4b-13de4e3543a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:54.542354 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b78ea-7777-4605-8f4b-13de4e3543a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:54.622084 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:54.645949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b78ea-7777-4605-8f4b-13de4e3543a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:54.647430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b78ea-7777-4605-8f4b-13de4e3543a6 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:54.658993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1139/2274] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:54.776668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d71cce-5f0c-4987-9a06-58a9c81d3829 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:54.778299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b0d71cce-5f0c-4987-9a06-58a9c81d3829 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:54.844327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d71cce-5f0c-4987-9a06-58a9c81d3829 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:54.848004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00184253-734f-46e8-8ce6-ec610ad33576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:54.848667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00184253-734f-46e8-8ce6-ec610ad33576 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:54.848960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00184253-734f-46e8-8ce6-ec610ad33576 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:54.849277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00184253-734f-46e8-8ce6-ec610ad33576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:54.851322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-00184253-734f-46e8-8ce6-ec610ad33576 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:54.928469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:05:54.931236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00184253-734f-46e8-8ce6-ec610ad33576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:54.933332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00184253-734f-46e8-8ce6-ec610ad33576 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:54.945392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1136/2275] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:05:55.261045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b78ea-7777-4605-8f4b-13de4e3543a6 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:05:55.262542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8d4b78ea-7777-4605-8f4b-13de4e3543a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:05:55.274932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b78ea-7777-4605-8f4b-13de4e3543a6 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:05:55.279261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ed65f-abe6-4ede-bb56-416a4f767dab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:05:55.279915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ed65f-abe6-4ede-bb56-416a4f767dab service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:05:55.280204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ed65f-abe6-4ede-bb56-416a4f767dab service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:05:55.280486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ed65f-abe6-4ede-bb56-416a4f767dab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:05:55.291014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ed65f-abe6-4ede-bb56-416a4f767dab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:05:55.292483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ed65f-abe6-4ede-bb56-416a4f767dab service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:05:55.309630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1140/2276] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:05:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:06:00.351759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00184253-734f-46e8-8ce6-ec610ad33576 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:00.427838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00184253-734f-46e8-8ce6-ec610ad33576 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:00.434006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:00.434932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:00.435243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:00.435499 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] PATH_INFO: `/v3/users/1bc4a71f23254779b0209a1954c6743e` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:00.437446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc4a71f23254779b0209a1954c6743e)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:00.438838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:00.506497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc4a71f23254779b0209a1954c6743e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:00.523526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc4a71f23254779b0209a1954c6743e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:00.538088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc4a71f23254779b0209a1954c6743e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:00.547218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc4a71f23254779b0209a1954c6743e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:00.548424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc4a71f23254779b0209a1954c6743e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:00.558903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1137/2277] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:06:00 2022] DELETE /identity/v3/users/1bc4a71f23254779b0209a1954c6743e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:06:00.561400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5a9c9-bf97-4e74-b9a3-3b72bbe52643 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:00.561967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5a9c9-bf97-4e74-b9a3-3b72bbe52643 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:00.562159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5a9c9-bf97-4e74-b9a3-3b72bbe52643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:00.586943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-97c5a9c9-bf97-4e74-b9a3-3b72bbe52643 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:00.588323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-97c5a9c9-bf97-4e74-b9a3-3b72bbe52643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:00.669031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1141/2278] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:00.680752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:00.682278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:00.776946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54fd53-57e1-4ae4-9f3c-f4d1c296dc96 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:00.780764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a19b4-e5e4-4f86-8f50-c547c5643552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:00.781454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a19b4-e5e4-4f86-8f50-c547c5643552 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:00.781740 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a19b4-e5e4-4f86-8f50-c547c5643552 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:00.782048 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a19b4-e5e4-4f86-8f50-c547c5643552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:00.797332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a19b4-e5e4-4f86-8f50-c547c5643552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:00.797332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a19b4-e5e4-4f86-8f50-c547c5643552 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:00.807196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1138/2279] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:06:01.348506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5a9c9-bf97-4e74-b9a3-3b72bbe52643 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:01.349980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-97c5a9c9-bf97-4e74-b9a3-3b72bbe52643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:01.378558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5a9c9-bf97-4e74-b9a3-3b72bbe52643 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:01.383621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:01.384498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:01.384780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:01.385057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] PATH_INFO: `/v3/projects/6f2fe178a0034089ab01c9dda18147c7` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:01.386862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2fe178a0034089ab01c9dda18147c7)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:01.388792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:01.445526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2fe178a0034089ab01c9dda18147c7', 'request_id': 'req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.452422 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2fe178a0034089ab01c9dda18147c7', 'request_id': 'req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.460067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2fe178a0034089ab01c9dda18147c7', 'request_id': 'req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.476961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] The token cache is being invalidate because project 6f2fe178a0034089ab01c9dda18147c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:06:01.477287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.477974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.480029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1142/2280] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:06:01 2022] DELETE /identity/v3/projects/6f2fe178a0034089ab01c9dda18147c7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:06:01.483731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a19b4-e5e4-4f86-8f50-c547c5643552 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:01.494975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a19b4-e5e4-4f86-8f50-c547c5643552 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:01.508047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:01.571592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:01.572473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:01.572753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:01.573030 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] PATH_INFO: `/v3/users/b172edc92a924524b0d0314f5febf69c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:01.574824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b172edc92a924524b0d0314f5febf69c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:01.577219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:01.639889 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b172edc92a924524b0d0314f5febf69c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.653083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b172edc92a924524b0d0314f5febf69c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.661725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b172edc92a924524b0d0314f5febf69c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.670465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b172edc92a924524b0d0314f5febf69c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.671246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b172edc92a924524b0d0314f5febf69c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:01.677566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1139/2281] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:06:01 2022] DELETE /identity/v3/users/b172edc92a924524b0d0314f5febf69c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:06:01.979987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:01.981521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:01.995063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2901fb-1cca-46fa-a702-5e0538dcbcd5 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:02.048039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:02.060857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:02.061787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:02.062077 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:02.062359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] PATH_INFO: `/v3/projects/1611410aab444d2a979a1e021f0f4dac` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:02.063653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1611410aab444d2a979a1e021f0f4dac)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:02.065940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:02.127266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1611410aab444d2a979a1e021f0f4dac', 'request_id': 'req-4b1bd870-1807-4257-b6f7-6256b3bf3238'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.134084 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1611410aab444d2a979a1e021f0f4dac', 'request_id': 'req-4b1bd870-1807-4257-b6f7-6256b3bf3238'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.145170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1611410aab444d2a979a1e021f0f4dac', 'request_id': 'req-4b1bd870-1807-4257-b6f7-6256b3bf3238'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.158856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] The token cache is being invalidate because project 1611410aab444d2a979a1e021f0f4dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:06:02.159170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.159840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.161492 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1143/2282] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:06:01 2022] DELETE /identity/v3/projects/1611410aab444d2a979a1e021f0f4dac => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:06:02.165625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:02.167298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:02.181342 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46ef58-b951-4903-87b7-f9ddd76828d3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:02.240683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:02.241593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:02.241965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:02.242268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] PATH_INFO: `/v3/users/55b62488f46f4948bbc3a1e0abdb2139` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:02.243525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b62488f46f4948bbc3a1e0abdb2139)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:02.246336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:02.290349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:02.297428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b62488f46f4948bbc3a1e0abdb2139'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.311001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b62488f46f4948bbc3a1e0abdb2139'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.320494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b62488f46f4948bbc3a1e0abdb2139'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.328402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b62488f46f4948bbc3a1e0abdb2139'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.329112 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e1e04b3f-6578-4baa-bfbf-f3af5b919867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b62488f46f4948bbc3a1e0abdb2139'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.335165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1140/2283] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:06:02 2022] DELETE /identity/v3/users/55b62488f46f4948bbc3a1e0abdb2139 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:06:02.634228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:02.635643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:02.651617 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bd870-1807-4257-b6f7-6256b3bf3238 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:02.710802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:02.711656 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:02.711942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:02.712221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] PATH_INFO: `/v3/projects/f9085219f9a24111bfff2a852b4fa419` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:02.713480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9085219f9a24111bfff2a852b4fa419)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:02.715875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:02.765581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9085219f9a24111bfff2a852b4fa419', 'request_id': 'req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.771803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9085219f9a24111bfff2a852b4fa419', 'request_id': 'req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.779861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9085219f9a24111bfff2a852b4fa419', 'request_id': 'req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.790385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] The token cache is being invalidate because project f9085219f9a24111bfff2a852b4fa419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:06:02.790675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.791289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-03aeb1a4-d91d-4213-8017-7b9121f0ebb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:02.793473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1144/2284] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:06:02 2022] DELETE /identity/v3/projects/f9085219f9a24111bfff2a852b4fa419 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:06:02.798249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba988157-9fce-45a1-944a-62912f14fee8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:02.798727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba988157-9fce-45a1-944a-62912f14fee8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:02.798834 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba988157-9fce-45a1-944a-62912f14fee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:02.819144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ba988157-9fce-45a1-944a-62912f14fee8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:02.820608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ba988157-9fce-45a1-944a-62912f14fee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:02.886828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1141/2285] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:02.894602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fada5-6c84-4c0d-a83c-fcdbe8710e63 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:02.894894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fada5-6c84-4c0d-a83c-fcdbe8710e63 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:02.895188 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fada5-6c84-4c0d-a83c-fcdbe8710e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:02.915661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-460fada5-6c84-4c0d-a83c-fcdbe8710e63 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:02.916882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-460fada5-6c84-4c0d-a83c-fcdbe8710e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:02.924828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1145/2286] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:06:02.929138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba988157-9fce-45a1-944a-62912f14fee8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:02.939007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba988157-9fce-45a1-944a-62912f14fee8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:02.944003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d99357-6390-4ed5-8af0-08bb01e4a225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:02.944671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d99357-6390-4ed5-8af0-08bb01e4a225 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:02.944954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d99357-6390-4ed5-8af0-08bb01e4a225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:02.945294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d99357-6390-4ed5-8af0-08bb01e4a225 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:02.946060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d99357-6390-4ed5-8af0-08bb01e4a225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:06:02.946368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d99357-6390-4ed5-8af0-08bb01e4a225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:02.947969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d99357-6390-4ed5-8af0-08bb01e4a225 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:02.961251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1142/2287] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:06:02.965575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fada5-6c84-4c0d-a83c-fcdbe8710e63 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:02.971634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:02.976429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fada5-6c84-4c0d-a83c-fcdbe8710e63 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:02.983320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65f72e-e8ee-4e95-9ccd-82f035f9b7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:02.984192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65f72e-e8ee-4e95-9ccd-82f035f9b7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:02.984546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65f72e-e8ee-4e95-9ccd-82f035f9b7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:02.985005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65f72e-e8ee-4e95-9ccd-82f035f9b7f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:02.985958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65f72e-e8ee-4e95-9ccd-82f035f9b7f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:02.987864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65f72e-e8ee-4e95-9ccd-82f035f9b7f0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:03.014388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1146/2288] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:06:02 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:06:03.018413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d99357-6390-4ed5-8af0-08bb01e4a225 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:03.029582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d99357-6390-4ed5-8af0-08bb01e4a225 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:03.095746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90285009-bc5c-4538-8f0e-5f0a39280d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:03.096332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90285009-bc5c-4538-8f0e-5f0a39280d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:03.096617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90285009-bc5c-4538-8f0e-5f0a39280d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:03.096897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90285009-bc5c-4538-8f0e-5f0a39280d82 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:03.097671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90285009-bc5c-4538-8f0e-5f0a39280d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:03.099795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90285009-bc5c-4538-8f0e-5f0a39280d82 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:03.122492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1143/2289] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:06:03 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:06:03.126865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65f72e-e8ee-4e95-9ccd-82f035f9b7f0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:03.137936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65f72e-e8ee-4e95-9ccd-82f035f9b7f0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:03.142823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f119f-a993-49b8-9dc7-9cdc7f18a825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:03.143488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f119f-a993-49b8-9dc7-9cdc7f18a825 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:03.143776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f119f-a993-49b8-9dc7-9cdc7f18a825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:03.144091 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f119f-a993-49b8-9dc7-9cdc7f18a825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:03.144826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f119f-a993-49b8-9dc7-9cdc7f18a825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:03.146456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f119f-a993-49b8-9dc7-9cdc7f18a825 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:03.160926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1147/2290] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:06:03.164566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90285009-bc5c-4538-8f0e-5f0a39280d82 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:03.175978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90285009-bc5c-4538-8f0e-5f0a39280d82 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:03.180733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243c638-5056-4263-9d59-d6bdfd67e40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:03.181500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c638-5056-4263-9d59-d6bdfd67e40b admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:03.181787 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c638-5056-4263-9d59-d6bdfd67e40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:03.182061 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c638-5056-4263-9d59-d6bdfd67e40b admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:03.182820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243c638-5056-4263-9d59-d6bdfd67e40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:03.185179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243c638-5056-4263-9d59-d6bdfd67e40b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:03.197165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1144/2291] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:06:03.201479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f119f-a993-49b8-9dc7-9cdc7f18a825 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:03.213016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f119f-a993-49b8-9dc7-9cdc7f18a825 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:03.217857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac56bc-e900-4144-a3e4-c518f95094a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:03.218736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac56bc-e900-4144-a3e4-c518f95094a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:03.219025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac56bc-e900-4144-a3e4-c518f95094a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:03.219304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac56bc-e900-4144-a3e4-c518f95094a2 admin admin] PATH_INFO: `/v3/projects/3ede6086ad76404a8ad65f0525ac4a53/users/6af4701349d44699a8f04e1c8313de9e/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:03.234577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac56bc-e900-4144-a3e4-c518f95094a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ede6086ad76404a8ad65f0525ac4a53, user_id=6af4701349d44699a8f04e1c8313de9e, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:03.237703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac56bc-e900-4144-a3e4-c518f95094a2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:03.255696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1148/2292] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:06:03 2022] PUT /identity/v3/projects/3ede6086ad76404a8ad65f0525ac4a53/users/6af4701349d44699a8f04e1c8313de9e/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:06:03.261053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab0ca2-ee21-497e-9a62-1f0bffa4b958 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:03.261394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab0ca2-ee21-497e-9a62-1f0bffa4b958 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:03.261869 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab0ca2-ee21-497e-9a62-1f0bffa4b958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:03.283505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1bab0ca2-ee21-497e-9a62-1f0bffa4b958 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:03.284770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1bab0ca2-ee21-497e-9a62-1f0bffa4b958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:03.303023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:03.349109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1145/2293] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:06:03.359343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac56bc-e900-4144-a3e4-c518f95094a2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:03.361125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-81ac56bc-e900-4144-a3e4-c518f95094a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:03.418897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac56bc-e900-4144-a3e4-c518f95094a2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:03.422625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976e848-bd50-45c7-9001-91d6f84b4927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:03.423403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976e848-bd50-45c7-9001-91d6f84b4927 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:03.423694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976e848-bd50-45c7-9001-91d6f84b4927 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:03.423977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976e848-bd50-45c7-9001-91d6f84b4927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:03.433387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976e848-bd50-45c7-9001-91d6f84b4927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:03.434822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976e848-bd50-45c7-9001-91d6f84b4927 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:03.444675 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1149/2294] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:06:04.106365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33b301-a8e1-495e-915f-83519e6acb21 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:04.106943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33b301-a8e1-495e-915f-83519e6acb21 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:04.106943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33b301-a8e1-495e-915f-83519e6acb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:04.137697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7b33b301-a8e1-495e-915f-83519e6acb21 None None] MFA Rules not processed for user `6af4701349d44699a8f04e1c8313de9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:04.138821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7b33b301-a8e1-495e-915f-83519e6acb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:04.244396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1146/2295] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:04.254235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976e848-bd50-45c7-9001-91d6f84b4927 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:04.255688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c976e848-bd50-45c7-9001-91d6f84b4927 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:04.340000 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976e848-bd50-45c7-9001-91d6f84b4927 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:04.343996 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33890326-04e0-48c9-9994-59590dc250ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:04.344677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33890326-04e0-48c9-9994-59590dc250ad service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:04.344939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33890326-04e0-48c9-9994-59590dc250ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:04.345251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33890326-04e0-48c9-9994-59590dc250ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:04.354717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33890326-04e0-48c9-9994-59590dc250ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:04.356738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33890326-04e0-48c9-9994-59590dc250ad service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:04.366320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1150/2296] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:06:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:06:06.813611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359ab3-1cc5-4f4c-907c-89eaadc3e457 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:06.814046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359ab3-1cc5-4f4c-907c-89eaadc3e457 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:06.814201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359ab3-1cc5-4f4c-907c-89eaadc3e457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:06.832199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5d359ab3-1cc5-4f4c-907c-89eaadc3e457 None None] MFA Rules not processed for user `6af4701349d44699a8f04e1c8313de9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:06.833452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5d359ab3-1cc5-4f4c-907c-89eaadc3e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:06.840383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1147/2297] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:14.361775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=74,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:16.826635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=52 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:28.678084 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547509-e98c-4030-b37d-9d4448e5d58d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:28.678817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547509-e98c-4030-b37d-9d4448e5d58d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:28.678817 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76547509-e98c-4030-b37d-9d4448e5d58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:28.698572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-76547509-e98c-4030-b37d-9d4448e5d58d None None] MFA Rules not processed for user `6af4701349d44699a8f04e1c8313de9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:28.699900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-76547509-e98c-4030-b37d-9d4448e5d58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:28.707903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1151/2298] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:31.163555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d359ab3-1cc5-4f4c-907c-89eaadc3e457 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:31.165402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5d359ab3-1cc5-4f4c-907c-89eaadc3e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:31.190872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d359ab3-1cc5-4f4c-907c-89eaadc3e457 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:31.198232 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:31.199305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:31.199566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:31.199886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] PATH_INFO: `/v3/users/51a0f0fe2e5643b88526a1dba45b8eb6` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:31.203189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a0f0fe2e5643b88526a1dba45b8eb6)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:31.205296 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:31.275746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a0f0fe2e5643b88526a1dba45b8eb6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.304801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a0f0fe2e5643b88526a1dba45b8eb6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.316160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a0f0fe2e5643b88526a1dba45b8eb6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.328556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a0f0fe2e5643b88526a1dba45b8eb6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.329660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a0f0fe2e5643b88526a1dba45b8eb6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.337461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1148/2299] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:06:31 2022] DELETE /identity/v3/users/51a0f0fe2e5643b88526a1dba45b8eb6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:06:31.343156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113800b-bbfc-4fdc-b7e0-d434046eeed4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:31.343649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113800b-bbfc-4fdc-b7e0-d434046eeed4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:31.343813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113800b-bbfc-4fdc-b7e0-d434046eeed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:31.363053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f113800b-bbfc-4fdc-b7e0-d434046eeed4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:31.364260 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f113800b-bbfc-4fdc-b7e0-d434046eeed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:31.432377 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1152/2300] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:31.443476 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:31.445185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:31.507696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfcbc6-499b-4c63-ae3c-2a64e266cf24 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:31.511406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccc66d-b3a0-4f08-9e4c-9e6be3274bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:31.512101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccc66d-b3a0-4f08-9e4c-9e6be3274bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:31.512383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccc66d-b3a0-4f08-9e4c-9e6be3274bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:31.512659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccc66d-b3a0-4f08-9e4c-9e6be3274bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:31.522441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccc66d-b3a0-4f08-9e4c-9e6be3274bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:31.523911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccc66d-b3a0-4f08-9e4c-9e6be3274bb3 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:31.534694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1149/2301] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:06:31.854077 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113800b-bbfc-4fdc-b7e0-d434046eeed4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:31.855993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f113800b-bbfc-4fdc-b7e0-d434046eeed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:31.872707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113800b-bbfc-4fdc-b7e0-d434046eeed4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:31.880960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:31.882273 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:31.882610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:31.883082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] PATH_INFO: `/v3/projects/1cbe7203160648d0a1a0a65e7f65fdf5` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:31.885398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cbe7203160648d0a1a0a65e7f65fdf5)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:31.887801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:31.945471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cbe7203160648d0a1a0a65e7f65fdf5', 'request_id': 'req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.951782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cbe7203160648d0a1a0a65e7f65fdf5', 'request_id': 'req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.960695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cbe7203160648d0a1a0a65e7f65fdf5', 'request_id': 'req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.972771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] The token cache is being invalidate because project 1cbe7203160648d0a1a0a65e7f65fdf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:06:31.973126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.973747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-20f0437a-7f3b-4d23-8d1a-4bc3a603d5de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:06:31.975265 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1153/2302] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:06:31 2022] DELETE /identity/v3/projects/1cbe7203160648d0a1a0a65e7f65fdf5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:06:31.980808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025a3cd-3a7d-4606-873a-a5650e27ff9a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:31.981773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025a3cd-3a7d-4606-873a-a5650e27ff9a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:31.981773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025a3cd-3a7d-4606-873a-a5650e27ff9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.015849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2025a3cd-3a7d-4606-873a-a5650e27ff9a None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:32.017188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2025a3cd-3a7d-4606-873a-a5650e27ff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:32.089013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1150/2303] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:06:32.101540 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95f743-08df-4cef-83e5-fd62573308f0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:32.101540 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95f743-08df-4cef-83e5-fd62573308f0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:32.101861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95f743-08df-4cef-83e5-fd62573308f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.126402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5b95f743-08df-4cef-83e5-fd62573308f0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:32.128002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5b95f743-08df-4cef-83e5-fd62573308f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:32.142865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1154/2304] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:32.147371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025a3cd-3a7d-4606-873a-a5650e27ff9a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:32.157859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025a3cd-3a7d-4606-873a-a5650e27ff9a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:32.162467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c640ebe-2728-402d-9f28-3db53c104002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:32.163127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c640ebe-2728-402d-9f28-3db53c104002 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:32.163416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c640ebe-2728-402d-9f28-3db53c104002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:32.163688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c640ebe-2728-402d-9f28-3db53c104002 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.164428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c640ebe-2728-402d-9f28-3db53c104002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:06:32.164752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c640ebe-2728-402d-9f28-3db53c104002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:32.166386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c640ebe-2728-402d-9f28-3db53c104002 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:32.179317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1151/2305] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:06:32.183602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95f743-08df-4cef-83e5-fd62573308f0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:32.195779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95f743-08df-4cef-83e5-fd62573308f0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:32.200924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a18c8-b686-4ab0-85d5-4f6e0b51bd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:32.201761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a18c8-b686-4ab0-85d5-4f6e0b51bd05 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:32.202864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a18c8-b686-4ab0-85d5-4f6e0b51bd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:32.203208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a18c8-b686-4ab0-85d5-4f6e0b51bd05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.203962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a18c8-b686-4ab0-85d5-4f6e0b51bd05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:32.206056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a18c8-b686-4ab0-85d5-4f6e0b51bd05 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:32.238070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1155/2306] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:06:32 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:06:32.242312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c640ebe-2728-402d-9f28-3db53c104002 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:32.249083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:32.256901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c640ebe-2728-402d-9f28-3db53c104002 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:32.330713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403270-d584-4871-8154-dd116834a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:32.331546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403270-d584-4871-8154-dd116834a6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:32.331932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403270-d584-4871-8154-dd116834a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:32.332249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403270-d584-4871-8154-dd116834a6e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.333211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403270-d584-4871-8154-dd116834a6e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:32.335922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403270-d584-4871-8154-dd116834a6e5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:32.361495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1152/2307] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:06:32 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:06:32.365628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a18c8-b686-4ab0-85d5-4f6e0b51bd05 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:32.376250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a18c8-b686-4ab0-85d5-4f6e0b51bd05 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:32.380940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0d61f-32f5-4c6b-a2aa-e74a5a5d2e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:32.381635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0d61f-32f5-4c6b-a2aa-e74a5a5d2e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:32.381925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0d61f-32f5-4c6b-a2aa-e74a5a5d2e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:32.382206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0d61f-32f5-4c6b-a2aa-e74a5a5d2e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.382932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0d61f-32f5-4c6b-a2aa-e74a5a5d2e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:32.384527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0d61f-32f5-4c6b-a2aa-e74a5a5d2e9f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:32.396519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1156/2308] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:06:32.400730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403270-d584-4871-8154-dd116834a6e5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:32.415813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403270-d584-4871-8154-dd116834a6e5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:32.423603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbee8c-94fb-4215-9d7e-f3530a244200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:32.424874 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbee8c-94fb-4215-9d7e-f3530a244200 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:32.425244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbee8c-94fb-4215-9d7e-f3530a244200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:32.426112 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbee8c-94fb-4215-9d7e-f3530a244200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.427667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbee8c-94fb-4215-9d7e-f3530a244200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:32.431725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbee8c-94fb-4215-9d7e-f3530a244200 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:32.444430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1153/2309] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:06:32.448855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0d61f-32f5-4c6b-a2aa-e74a5a5d2e9f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:32.460202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0d61f-32f5-4c6b-a2aa-e74a5a5d2e9f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:32.465168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65d95-4a7f-472f-a28a-c208d6c52478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:32.466175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65d95-4a7f-472f-a28a-c208d6c52478 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:32.466462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65d95-4a7f-472f-a28a-c208d6c52478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:32.466765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65d95-4a7f-472f-a28a-c208d6c52478 admin admin] PATH_INFO: `/v3/projects/f9f8ecb9ed854cf28de16f950daf39e5/users/60db31dcc6944f63b078d2893640e0b8/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.483620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f65d95-4a7f-472f-a28a-c208d6c52478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f8ecb9ed854cf28de16f950daf39e5, user_id=60db31dcc6944f63b078d2893640e0b8, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:32.486718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f65d95-4a7f-472f-a28a-c208d6c52478 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:32.501120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1157/2310] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:06:32 2022] PUT /identity/v3/projects/f9f8ecb9ed854cf28de16f950daf39e5/users/60db31dcc6944f63b078d2893640e0b8/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:06:32.506767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee01fac-710c-4f95-9555-beab4b3d802e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:32.507059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee01fac-710c-4f95-9555-beab4b3d802e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:32.507345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee01fac-710c-4f95-9555-beab4b3d802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.527412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3ee01fac-710c-4f95-9555-beab4b3d802e None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:32.528651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3ee01fac-710c-4f95-9555-beab4b3d802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:32.599925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1154/2311] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:32.609957 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65d95-4a7f-472f-a28a-c208d6c52478 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:32.612432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-02f65d95-4a7f-472f-a28a-c208d6c52478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:32.631531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:32.684916 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65d95-4a7f-472f-a28a-c208d6c52478 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:32.689348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ced0c4-2ea5-405d-a24c-8f88fdd65831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:32.689995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ced0c4-2ea5-405d-a24c-8f88fdd65831 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:32.690528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ced0c4-2ea5-405d-a24c-8f88fdd65831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:32.690816 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ced0c4-2ea5-405d-a24c-8f88fdd65831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:32.700359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ced0c4-2ea5-405d-a24c-8f88fdd65831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:32.701919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ced0c4-2ea5-405d-a24c-8f88fdd65831 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:32.713366 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1158/2312] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:06:33.587139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b54141-02de-4f1f-b252-26260612efc8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:33.588059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b54141-02de-4f1f-b252-26260612efc8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:33.588059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b54141-02de-4f1f-b252-26260612efc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:33.629501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-66b54141-02de-4f1f-b252-26260612efc8 None None] MFA Rules not processed for user `60db31dcc6944f63b078d2893640e0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:33.630775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-66b54141-02de-4f1f-b252-26260612efc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:33.731861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1155/2313] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:33.743771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ced0c4-2ea5-405d-a24c-8f88fdd65831 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:33.745989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-84ced0c4-2ea5-405d-a24c-8f88fdd65831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:33.830601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ced0c4-2ea5-405d-a24c-8f88fdd65831 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:33.836169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435be4b-42fa-4e4e-903c-4b4e12b9d9b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:33.837139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435be4b-42fa-4e4e-903c-4b4e12b9d9b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:33.837461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435be4b-42fa-4e4e-903c-4b4e12b9d9b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:33.837983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435be4b-42fa-4e4e-903c-4b4e12b9d9b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:33.851112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435be4b-42fa-4e4e-903c-4b4e12b9d9b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:33.853038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435be4b-42fa-4e4e-903c-4b4e12b9d9b5 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:33.870872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1159/2314] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:06:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:06:35.335919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f85d3-217b-4b72-b8a5-5a420371227f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:35.336542 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f85d3-217b-4b72-b8a5-5a420371227f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:35.336542 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f85d3-217b-4b72-b8a5-5a420371227f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:35.354788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-203f85d3-217b-4b72-b8a5-5a420371227f None None] MFA Rules not processed for user `60db31dcc6944f63b078d2893640e0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:06:35.355926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-203f85d3-217b-4b72-b8a5-5a420371227f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:35.364301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1156/2315] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:06:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:06:43.864548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=46 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:45.349278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:06:54.371459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435be4b-42fa-4e4e-903c-4b4e12b9d9b5 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:06:54.372977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9435be4b-42fa-4e4e-903c-4b4e12b9d9b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:54.386308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435be4b-42fa-4e4e-903c-4b4e12b9d9b5 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:06:54.391132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:06:54.391880 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:06:54.392167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:06:54.392641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:06:54.395539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:06:54.473743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:06:54.475221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:06:54.486981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1160/2316] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:06:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:07:03.157012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2736c9-81e5-4261-b2de-ba5138cd4b48 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:03.157562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2736c9-81e5-4261-b2de-ba5138cd4b48 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:03.157707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2736c9-81e5-4261-b2de-ba5138cd4b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:03.224831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-be2736c9-81e5-4261-b2de-ba5138cd4b48 None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:07:03.226281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-be2736c9-81e5-4261-b2de-ba5138cd4b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:03.293322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1157/2317] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:07:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:07:03.337129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:03.338616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:03.439225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5311a-b2cc-4748-b09a-e96581f28910 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:03.444699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2f433-9907-41e8-a35e-76155ed72011 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:03.445381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2f433-9907-41e8-a35e-76155ed72011 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:03.445672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2f433-9907-41e8-a35e-76155ed72011 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:03.445956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2f433-9907-41e8-a35e-76155ed72011 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:03.455157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2f433-9907-41e8-a35e-76155ed72011 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:03.456741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2f433-9907-41e8-a35e-76155ed72011 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:03.466072 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1161/2318] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:07:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:07:13.290209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=44 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:07:13.462913 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=71 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:07:25.968679 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9237b-d7e6-42c4-92f3-7da970b6f305 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:25.969364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9237b-d7e6-42c4-92f3-7da970b6f305 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:25.969537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9237b-d7e6-42c4-92f3-7da970b6f305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:25.990433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-89f9237b-d7e6-42c4-92f3-7da970b6f305 None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:07:25.991737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-89f9237b-d7e6-42c4-92f3-7da970b6f305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:26.006629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1158/2319] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:07:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:07:26.030001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2f433-9907-41e8-a35e-76155ed72011 service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:26.049349 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2f433-9907-41e8-a35e-76155ed72011 service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:26.057835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce78a1-413a-4cf3-bcbc-fa238fe0c24a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:26.058724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce78a1-413a-4cf3-bcbc-fa238fe0c24a service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:26.059238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce78a1-413a-4cf3-bcbc-fa238fe0c24a service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:26.059822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce78a1-413a-4cf3-bcbc-fa238fe0c24a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:26.075908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce78a1-413a-4cf3-bcbc-fa238fe0c24a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:26.077932 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce78a1-413a-4cf3-bcbc-fa238fe0c24a service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:26.090509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1162/2320] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:07:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:07:28.156008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c696b9-49cf-4a07-ab1f-23bba2529fd2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:28.157179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c696b9-49cf-4a07-ab1f-23bba2529fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:28.157752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c696b9-49cf-4a07-ab1f-23bba2529fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:28.175472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-90c696b9-49cf-4a07-ab1f-23bba2529fd2 None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:07:28.176679 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-90c696b9-49cf-4a07-ab1f-23bba2529fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:28.183978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1159/2321] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jun 21 19:07:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:07:28.198989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce78a1-413a-4cf3-bcbc-fa238fe0c24a service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:28.212319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce78a1-413a-4cf3-bcbc-fa238fe0c24a service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:28.217642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7825454-7d51-4444-844a-a6e1e60ad28a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:28.218335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7825454-7d51-4444-844a-a6e1e60ad28a service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:28.218630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7825454-7d51-4444-844a-a6e1e60ad28a service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:28.218910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7825454-7d51-4444-844a-a6e1e60ad28a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:28.232084 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7825454-7d51-4444-844a-a6e1e60ad28a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:28.233891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7825454-7d51-4444-844a-a6e1e60ad28a service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:28.244271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1163/2322] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:07:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:07:30.410980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c696b9-49cf-4a07-ab1f-23bba2529fd2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:30.413925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-90c696b9-49cf-4a07-ab1f-23bba2529fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:30.443208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c696b9-49cf-4a07-ab1f-23bba2529fd2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:30.452782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:30.453726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:30.454054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:30.454411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] PATH_INFO: `/v3/users/6af4701349d44699a8f04e1c8313de9e` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:30.455666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6af4701349d44699a8f04e1c8313de9e)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:30.457710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:30.517421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6af4701349d44699a8f04e1c8313de9e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:30.530907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6af4701349d44699a8f04e1c8313de9e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:30.539657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6af4701349d44699a8f04e1c8313de9e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:30.547710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6af4701349d44699a8f04e1c8313de9e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:30.548403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6af4701349d44699a8f04e1c8313de9e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:30.554553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1160/2323] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:07:30 2022] DELETE /identity/v3/users/6af4701349d44699a8f04e1c8313de9e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:07:30.818835 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7825454-7d51-4444-844a-a6e1e60ad28a service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:30.832810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7825454-7d51-4444-844a-a6e1e60ad28a service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:30.925167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:30.926096 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:30.926384 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:30.926696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] PATH_INFO: `/v3/projects/3ede6086ad76404a8ad65f0525ac4a53` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:30.928305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ede6086ad76404a8ad65f0525ac4a53)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:30.930427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:30.988766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ede6086ad76404a8ad65f0525ac4a53', 'request_id': 'req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:30.997158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ede6086ad76404a8ad65f0525ac4a53', 'request_id': 'req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:31.011075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ede6086ad76404a8ad65f0525ac4a53', 'request_id': 'req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:31.024613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] The token cache is being invalidate because project 3ede6086ad76404a8ad65f0525ac4a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:07:31.025234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:31.026066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:07:31.028889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1164/2324] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:07:30 2022] DELETE /identity/v3/projects/3ede6086ad76404a8ad65f0525ac4a53 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:07:36.405618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:36.407908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:36.483522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5155f18-7070-4fac-a44a-b3d9a82bb757 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:36.573665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386d8cd-1b7c-42d9-aa6e-1842504a6223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:36.575421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386d8cd-1b7c-42d9-aa6e-1842504a6223 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:36.576265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386d8cd-1b7c-42d9-aa6e-1842504a6223 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:36.577188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386d8cd-1b7c-42d9-aa6e-1842504a6223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:36.580871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f386d8cd-1b7c-42d9-aa6e-1842504a6223 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:36.632347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:07:36.694706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386d8cd-1b7c-42d9-aa6e-1842504a6223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:36.696490 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386d8cd-1b7c-42d9-aa6e-1842504a6223 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:36.709522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1161/2325] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:07:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:07:41.020963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:07:41.875214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:41.876640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:41.935963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4f822-34d6-4faa-b0b9-9cc99e794ed2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:41.939572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6589a-6067-4442-b1b0-fb4831abd393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:41.940480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6589a-6067-4442-b1b0-fb4831abd393 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:41.940765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6589a-6067-4442-b1b0-fb4831abd393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:41.941044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6589a-6067-4442-b1b0-fb4831abd393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:41.942847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-11f6589a-6067-4442-b1b0-fb4831abd393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:41.956882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6589a-6067-4442-b1b0-fb4831abd393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:41.958373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6589a-6067-4442-b1b0-fb4831abd393 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:41.970139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1165/2326] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:07:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:07:45.000386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e6a3c-adc2-4ce6-83ab-dec5349f2d67 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:45.001330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e6a3c-adc2-4ce6-83ab-dec5349f2d67 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:45.001638 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e6a3c-adc2-4ce6-83ab-dec5349f2d67 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:45.003461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1162/2327] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 19:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:07:45.010986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635b1b3-34f6-4f8d-851b-ab419ea4ade6 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:45.011650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635b1b3-34f6-4f8d-851b-ab419ea4ade6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:45.011981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635b1b3-34f6-4f8d-851b-ab419ea4ade6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:45.068929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2635b1b3-34f6-4f8d-851b-ab419ea4ade6 None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:07:45.068929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2635b1b3-34f6-4f8d-851b-ab419ea4ade6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:45.079647 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1166/2328] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:07:45.093966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e6a3c-adc2-4ce6-83ab-dec5349f2d67 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:45.096254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-008e6a3c-adc2-4ce6-83ab-dec5349f2d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:45.191401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e6a3c-adc2-4ce6-83ab-dec5349f2d67 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:45.195481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22b5e6-41b6-45a1-97e4-ae38a5fad9ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915704f26c814eaa9e6c2a68533a9c66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:45.196208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22b5e6-41b6-45a1-97e4-ae38a5fad9ec service glance] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:45.196647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22b5e6-41b6-45a1-97e4-ae38a5fad9ec service glance] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:45.196961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22b5e6-41b6-45a1-97e4-ae38a5fad9ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:45.209683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22b5e6-41b6-45a1-97e4-ae38a5fad9ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:45.211178 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22b5e6-41b6-45a1-97e4-ae38a5fad9ec service glance] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:45.222935 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1163/2329] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:07:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:07:45.296255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3994a3-25cd-465b-a38b-25077bf7ecb6 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:45.296559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3994a3-25cd-465b-a38b-25077bf7ecb6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:45.296839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3994a3-25cd-465b-a38b-25077bf7ecb6 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:45.298278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1167/2330] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 19:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:07:45.303139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164cb6a-c797-4491-84df-193a3048034a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:45.303451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164cb6a-c797-4491-84df-193a3048034a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:45.303739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164cb6a-c797-4491-84df-193a3048034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:45.325468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f164cb6a-c797-4491-84df-193a3048034a None None] MFA Rules not processed for user `808b530d24b3487b9a20ee74c131502e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:07:45.326498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f164cb6a-c797-4491-84df-193a3048034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:45.333377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1164/2331] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:07:45.345865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3994a3-25cd-465b-a38b-25077bf7ecb6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:45.348175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fd3994a3-25cd-465b-a38b-25077bf7ecb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:45.361956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3994a3-25cd-465b-a38b-25077bf7ecb6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:45.365639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ac2f1-f349-4721-a79f-5a67067ca299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:45.366296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ac2f1-f349-4721-a79f-5a67067ca299 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:45.366584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ac2f1-f349-4721-a79f-5a67067ca299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:45.366864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ac2f1-f349-4721-a79f-5a67067ca299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:45.375001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ac2f1-f349-4721-a79f-5a67067ca299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:45.376437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ac2f1-f349-4721-a79f-5a67067ca299 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:45.385718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1168/2332] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:07:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:07:45.473595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a69f8-4c19-48e6-9c34-299c81251db2 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:45.473913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a69f8-4c19-48e6-9c34-299c81251db2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:45.474195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a69f8-4c19-48e6-9c34-299c81251db2 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:45.475708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1165/2333] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 19:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:07:45.481571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704b94d-2f77-4dab-ab12-8ffec7341333 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:45.481972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704b94d-2f77-4dab-ab12-8ffec7341333 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:45.482347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704b94d-2f77-4dab-ab12-8ffec7341333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:45.502216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1704b94d-2f77-4dab-ab12-8ffec7341333 None None] MFA Rules not processed for user `dcbcec65220c4ebb95f974925fa73f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:07:45.503443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1704b94d-2f77-4dab-ab12-8ffec7341333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:45.513696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1169/2334] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 19:07:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:07:45.525542 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a69f8-4c19-48e6-9c34-299c81251db2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:45.536837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a69f8-4c19-48e6-9c34-299c81251db2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:45.540778 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cdbc8-02d5-4ec4-8aa3-a8430d18ba43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:45.541401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cdbc8-02d5-4ec4-8aa3-a8430d18ba43 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:45.541740 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cdbc8-02d5-4ec4-8aa3-a8430d18ba43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:45.541970 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cdbc8-02d5-4ec4-8aa3-a8430d18ba43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:45.554539 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cdbc8-02d5-4ec4-8aa3-a8430d18ba43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:45.555669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cdbc8-02d5-4ec4-8aa3-a8430d18ba43 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:45.570078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1166/2335] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:07:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:07:51.497593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704b94d-2f77-4dab-ab12-8ffec7341333 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:51.499020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1704b94d-2f77-4dab-ab12-8ffec7341333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:51.510848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704b94d-2f77-4dab-ab12-8ffec7341333 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:51.515810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:51.516491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:51.516818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:51.517221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:51.519172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:51.577895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:51.579267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:51.588404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1170/2336] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:07:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:07:55.563231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=64 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:07:56.653852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648712-7e58-4c36-9a03-e78b898fead2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:56.654477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648712-7e58-4c36-9a03-e78b898fead2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:56.655497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648712-7e58-4c36-9a03-e78b898fead2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:56.673154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-21648712-7e58-4c36-9a03-e78b898fead2 None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:07:56.674378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-21648712-7e58-4c36-9a03-e78b898fead2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:56.732885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1167/2337] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:07:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:07:56.746945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:07:56.748404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:07:56.816881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:07:56.821703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ae35-9ebd-4959-9413-47c75bad1b2c service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:07:56.827109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15f87f-570d-4d48-ab94-20b0063f682d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:07:56.827754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15f87f-570d-4d48-ab94-20b0063f682d service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:07:56.828036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15f87f-570d-4d48-ab94-20b0063f682d service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:07:56.828309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15f87f-570d-4d48-ab94-20b0063f682d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:07:56.837161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15f87f-570d-4d48-ab94-20b0063f682d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:07:56.838840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15f87f-570d-4d48-ab94-20b0063f682d service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:07:56.848634 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1171/2338] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:07:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:08:06.725875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=25 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:08:06.845014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:08:12.406314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21648712-7e58-4c36-9a03-e78b898fead2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:08:12.420227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21648712-7e58-4c36-9a03-e78b898fead2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:08:12.425265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168d074-66d4-4756-a06a-7e6f4258107b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:08:12.426212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168d074-66d4-4756-a06a-7e6f4258107b service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:08:12.426649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168d074-66d4-4756-a06a-7e6f4258107b service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:08:12.427081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168d074-66d4-4756-a06a-7e6f4258107b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:08:12.429648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7168d074-66d4-4756-a06a-7e6f4258107b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:12.442640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168d074-66d4-4756-a06a-7e6f4258107b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:08:12.444328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168d074-66d4-4756-a06a-7e6f4258107b service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:08:12.453831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1168/2339] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:08:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:08:22.449563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:08:29.179251 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15f87f-570d-4d48-ab94-20b0063f682d service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:08:29.180677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1e15f87f-570d-4d48-ab94-20b0063f682d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:29.196250 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15f87f-570d-4d48-ab94-20b0063f682d service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:08:29.202662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e81574-425a-4d19-9d26-4799f552eebe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:08:29.203319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e81574-425a-4d19-9d26-4799f552eebe service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:08:29.203668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e81574-425a-4d19-9d26-4799f552eebe service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:08:29.204090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e81574-425a-4d19-9d26-4799f552eebe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:08:29.205912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-04e81574-425a-4d19-9d26-4799f552eebe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:29.267095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e81574-425a-4d19-9d26-4799f552eebe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:08:29.268596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e81574-425a-4d19-9d26-4799f552eebe service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:08:29.278567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1172/2340] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:08:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:08:33.977604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae427fc4-f41f-4bea-aaf9-09f4834784d1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:08:33.978386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae427fc4-f41f-4bea-aaf9-09f4834784d1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:08:33.978386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae427fc4-f41f-4bea-aaf9-09f4834784d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:08:33.997626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ae427fc4-f41f-4bea-aaf9-09f4834784d1 None None] MFA Rules not processed for user `60db31dcc6944f63b078d2893640e0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:08:33.998964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ae427fc4-f41f-4bea-aaf9-09f4834784d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:34.067958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1169/2341] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:08:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:08:34.341943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e81574-425a-4d19-9d26-4799f552eebe service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:08:34.356648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e81574-425a-4d19-9d26-4799f552eebe service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:08:34.362829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bc6b6-58fe-4e2b-ae26-81ac0110aee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:08:34.362829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bc6b6-58fe-4e2b-ae26-81ac0110aee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:08:34.362829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bc6b6-58fe-4e2b-ae26-81ac0110aee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:08:34.362829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bc6b6-58fe-4e2b-ae26-81ac0110aee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:08:34.365431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-057bc6b6-58fe-4e2b-ae26-81ac0110aee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:34.378212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057bc6b6-58fe-4e2b-ae26-81ac0110aee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:08:34.379645 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057bc6b6-58fe-4e2b-ae26-81ac0110aee0 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:08:34.390014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1173/2342] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:08:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:08:34.789979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae427fc4-f41f-4bea-aaf9-09f4834784d1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:08:34.791397 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ae427fc4-f41f-4bea-aaf9-09f4834784d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:34.865280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae427fc4-f41f-4bea-aaf9-09f4834784d1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:08:34.870542 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e246f4-d90b-4baf-a356-670096077898 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:08:34.871192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e246f4-d90b-4baf-a356-670096077898 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:08:34.871480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e246f4-d90b-4baf-a356-670096077898 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:08:34.871758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e246f4-d90b-4baf-a356-670096077898 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:08:34.873581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-53e246f4-d90b-4baf-a356-670096077898 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:34.886330 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e246f4-d90b-4baf-a356-670096077898 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:08:34.888004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e246f4-d90b-4baf-a356-670096077898 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:08:34.903010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1170/2343] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:08:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:08:34.941446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722c7e6-f830-419c-8817-016e71e6c91a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:08:34.941825 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722c7e6-f830-419c-8817-016e71e6c91a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:08:34.942173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722c7e6-f830-419c-8817-016e71e6c91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:08:34.963542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e722c7e6-f830-419c-8817-016e71e6c91a None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:08:34.964873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e722c7e6-f830-419c-8817-016e71e6c91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:34.973951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1174/2344] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jun 21 19:08:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:08:34.999079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e246f4-d90b-4baf-a356-670096077898 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:08:35.020215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e246f4-d90b-4baf-a356-670096077898 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:08:35.025681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d609d9d-514f-45f4-83b0-e1e60fcca08e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:08:35.026354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d609d9d-514f-45f4-83b0-e1e60fcca08e service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:08:35.026639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d609d9d-514f-45f4-83b0-e1e60fcca08e service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:08:35.026926 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d609d9d-514f-45f4-83b0-e1e60fcca08e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:08:35.037753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d609d9d-514f-45f4-83b0-e1e60fcca08e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:08:35.039389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d609d9d-514f-45f4-83b0-e1e60fcca08e service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:08:35.049533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1171/2345] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:08:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:08:37.612259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722c7e6-f830-419c-8817-016e71e6c91a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:08:37.628847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722c7e6-f830-419c-8817-016e71e6c91a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:08:37.636823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee70684-6b71-406d-b937-37d3d0e5c896 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:08:37.637578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee70684-6b71-406d-b937-37d3d0e5c896 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:08:37.638006 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee70684-6b71-406d-b937-37d3d0e5c896 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:08:37.638201 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee70684-6b71-406d-b937-37d3d0e5c896 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:08:37.640456 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9ee70684-6b71-406d-b937-37d3d0e5c896 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:37.655782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee70684-6b71-406d-b937-37d3d0e5c896 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:08:37.657454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee70684-6b71-406d-b937-37d3d0e5c896 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:08:37.670233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1175/2346] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:08:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:08:45.045729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=63 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:08:45.176577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d609d9d-514f-45f4-83b0-e1e60fcca08e service swift] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:08:45.193300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d609d9d-514f-45f4-83b0-e1e60fcca08e service swift] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:08:45.200130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f0eb-6eea-4ae9-989e-b2907d30c4f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:08:45.200903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f0eb-6eea-4ae9-989e-b2907d30c4f0 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:08:45.201300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f0eb-6eea-4ae9-989e-b2907d30c4f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:08:45.201653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f0eb-6eea-4ae9-989e-b2907d30c4f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:08:45.204304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0e29f0eb-6eea-4ae9-989e-b2907d30c4f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:08:45.219096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f0eb-6eea-4ae9-989e-b2907d30c4f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:08:45.220768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f0eb-6eea-4ae9-989e-b2907d30c4f0 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:08:45.234040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1172/2347] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:08:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:08:47.664555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=43 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:08:55.228950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:09:05.821571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da8754-8e76-480f-8328-51c1f70721af None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:09:05.822327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da8754-8e76-480f-8328-51c1f70721af None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:09:05.822327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da8754-8e76-480f-8328-51c1f70721af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:09:05.839922 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f6da8754-8e76-480f-8328-51c1f70721af None None] MFA Rules not processed for user `569256e6d1d1418eafb5ff2ed8909926`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:09:05.841167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f6da8754-8e76-480f-8328-51c1f70721af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:09:05.899982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1176/2348] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:09:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:09:15.560542 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44859f85-8618-49db-9d61-a517346bcfb8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:09:15.561174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44859f85-8618-49db-9d61-a517346bcfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:09:15.561174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44859f85-8618-49db-9d61-a517346bcfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:09:15.582078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-44859f85-8618-49db-9d61-a517346bcfb8 None None] MFA Rules not processed for user `eeafe22d22f648d6aefcc6a308d4fc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:09:15.583509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-44859f85-8618-49db-9d61-a517346bcfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:09:15.591094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1173/2349] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:09:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:09:15.892921 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=25 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:09:25.574044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:09:29.930939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da8754-8e76-480f-8328-51c1f70721af None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:09:29.932597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f6da8754-8e76-480f-8328-51c1f70721af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:09:29.949286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da8754-8e76-480f-8328-51c1f70721af None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:09:29.955084 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:09:29.955784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:09:29.956066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:09:29.956345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:09:29.958248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:09:30.032475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:09:30.034020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:09:30.045275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1177/2350] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:09:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:09:40.039935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=28 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:09:51.762104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44859f85-8618-49db-9d61-a517346bcfb8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:09:51.775715 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44859f85-8618-49db-9d61-a517346bcfb8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:09:51.780932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3940f4a-ab93-45ce-be9c-56e11d9f7045 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:09:51.782037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3940f4a-ab93-45ce-be9c-56e11d9f7045 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:09:51.782037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3940f4a-ab93-45ce-be9c-56e11d9f7045 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:09:51.782265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3940f4a-ab93-45ce-be9c-56e11d9f7045 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:09:51.784082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d3940f4a-ab93-45ce-be9c-56e11d9f7045 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:09:51.796129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3940f4a-ab93-45ce-be9c-56e11d9f7045 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:09:51.797774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3940f4a-ab93-45ce-be9c-56e11d9f7045 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:09:51.807920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1174/2351] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:09:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:09:51.836861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:09:51.838413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:09:51.904225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f744b29-b0a5-48d2-925e-d029b6421d2d service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:09:51.908717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:09:51.909610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:09:51.909898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:09:51.910176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] PATH_INFO: `/v3/users/60db31dcc6944f63b078d2893640e0b8` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:09:51.911389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=60db31dcc6944f63b078d2893640e0b8)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:09:51.913380 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:09:51.962533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60db31dcc6944f63b078d2893640e0b8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:51.974180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60db31dcc6944f63b078d2893640e0b8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:51.981358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60db31dcc6944f63b078d2893640e0b8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:51.982075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60db31dcc6944f63b078d2893640e0b8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:51.988967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc1eb0ff-7b3b-47db-b0cb-74fc5de3e8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60db31dcc6944f63b078d2893640e0b8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:51.995717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1178/2352] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:09:51 2022] DELETE /identity/v3/users/60db31dcc6944f63b078d2893640e0b8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:09:52.174100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3940f4a-ab93-45ce-be9c-56e11d9f7045 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:09:52.184915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3940f4a-ab93-45ce-be9c-56e11d9f7045 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:09:52.246134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:09:52.246911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:09:52.247199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:09:52.247483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] PATH_INFO: `/v3/projects/f9f8ecb9ed854cf28de16f950daf39e5` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:09:52.248683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f8ecb9ed854cf28de16f950daf39e5)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:09:52.250631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:09:52.297583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f8ecb9ed854cf28de16f950daf39e5', 'request_id': 'req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:52.304181 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f8ecb9ed854cf28de16f950daf39e5', 'request_id': 'req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:52.312271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f8ecb9ed854cf28de16f950daf39e5', 'request_id': 'req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:52.327117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] The token cache is being invalidate because project f9f8ecb9ed854cf28de16f950daf39e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:09:52.327515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:52.328253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:09:52.329913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1175/2353] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:09:52 2022] DELETE /identity/v3/projects/f9f8ecb9ed854cf28de16f950daf39e5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:10:01.991451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:10:02.323047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:10:17.340736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e18394-2f35-494f-bcb9-f2321fef27f8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:17.341309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e18394-2f35-494f-bcb9-f2321fef27f8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:17.341309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e18394-2f35-494f-bcb9-f2321fef27f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:17.360313 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c8e18394-2f35-494f-bcb9-f2321fef27f8 None None] MFA Rules not processed for user `b6ce650c0ae6494ab895644347bbb5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:10:17.361705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c8e18394-2f35-494f-bcb9-f2321fef27f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:17.419884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1179/2354] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:10:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:10:27.413083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=25 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:10:47.635671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:47.637071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:47.709561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db3be8-3eaf-4a8b-96fa-ce41a467c0f1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:47.714891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:47.715569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:47.715890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:47.716175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:47.717996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:47.730291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:47.731898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:47.742566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1176/2355] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:10:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:10:50.886802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e18394-2f35-494f-bcb9-f2321fef27f8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:50.888528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c8e18394-2f35-494f-bcb9-f2321fef27f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:50.945999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e18394-2f35-494f-bcb9-f2321fef27f8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:50.949599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589349e-af7b-4eef-a63d-f6ee128363ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:50.950239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589349e-af7b-4eef-a63d-f6ee128363ee service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:50.950521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589349e-af7b-4eef-a63d-f6ee128363ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:50.950797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589349e-af7b-4eef-a63d-f6ee128363ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:50.953155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1589349e-af7b-4eef-a63d-f6ee128363ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:51.014684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589349e-af7b-4eef-a63d-f6ee128363ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:51.016116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589349e-af7b-4eef-a63d-f6ee128363ee service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:51.026421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1180/2356] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:10:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:10:54.245256 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:54.246780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:54.264915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248384d-25ec-4d40-854b-3ccba2d39174 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:54.269978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:54.270863 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:54.271151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:54.271434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] PATH_INFO: `/v3/users/eeafe22d22f648d6aefcc6a308d4fc0d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:54.272654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeafe22d22f648d6aefcc6a308d4fc0d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:54.274623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:54.327624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeafe22d22f648d6aefcc6a308d4fc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.341359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeafe22d22f648d6aefcc6a308d4fc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.349801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeafe22d22f648d6aefcc6a308d4fc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.357483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeafe22d22f648d6aefcc6a308d4fc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.358152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf2f9a62-16a7-49f4-aa70-c8ceb091daf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeafe22d22f648d6aefcc6a308d4fc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.365458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1177/2357] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:10:54 2022] DELETE /identity/v3/users/eeafe22d22f648d6aefcc6a308d4fc0d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:10:54.558484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589349e-af7b-4eef-a63d-f6ee128363ee service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:54.569183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589349e-af7b-4eef-a63d-f6ee128363ee service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:54.628356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:54.629255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:54.629578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:54.629861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] PATH_INFO: `/v3/projects/f1af5239143346e8b995f4176d78ca4d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:54.631067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1af5239143346e8b995f4176d78ca4d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:54.632969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:54.688134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1af5239143346e8b995f4176d78ca4d', 'request_id': 'req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.690430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:10:54.695288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1af5239143346e8b995f4176d78ca4d', 'request_id': 'req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.702581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1af5239143346e8b995f4176d78ca4d', 'request_id': 'req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.713116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] The token cache is being invalidate because project f1af5239143346e8b995f4176d78ca4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:10:54.713423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.714063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89c2c9b-4134-42bf-b7bf-26ddcc9c10a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:10:54.715604 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1181/2358] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:10:54 2022] DELETE /identity/v3/projects/f1af5239143346e8b995f4176d78ca4d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:10:54.721111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fb8a6-4349-47a7-a0f2-f536300cd0ad None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:54.721404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fb8a6-4349-47a7-a0f2-f536300cd0ad None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:54.721741 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fb8a6-4349-47a7-a0f2-f536300cd0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:54.741207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-905fb8a6-4349-47a7-a0f2-f536300cd0ad None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:10:54.742516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-905fb8a6-4349-47a7-a0f2-f536300cd0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:54.803479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1178/2359] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:10:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:10:54.811062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a887d7-b9b5-44e4-875e-b51c22cf9e1b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:54.811382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a887d7-b9b5-44e4-875e-b51c22cf9e1b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:54.811666 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a887d7-b9b5-44e4-875e-b51c22cf9e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:54.830429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-65a887d7-b9b5-44e4-875e-b51c22cf9e1b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:10:54.831677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-65a887d7-b9b5-44e4-875e-b51c22cf9e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:54.839769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1182/2360] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:10:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:10:54.843925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fb8a6-4349-47a7-a0f2-f536300cd0ad None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:54.849269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:10:54.855802 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fb8a6-4349-47a7-a0f2-f536300cd0ad None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:54.861509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97f057-8bb5-4ddf-a07d-c6aad469fdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:54.862174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f057-8bb5-4ddf-a07d-c6aad469fdab admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:54.862462 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f057-8bb5-4ddf-a07d-c6aad469fdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:54.862747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f057-8bb5-4ddf-a07d-c6aad469fdab admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:54.863486 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97f057-8bb5-4ddf-a07d-c6aad469fdab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:10:54.863845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97f057-8bb5-4ddf-a07d-c6aad469fdab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:54.866018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97f057-8bb5-4ddf-a07d-c6aad469fdab admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:54.877963 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1179/2361] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:10:54.882291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a887d7-b9b5-44e4-875e-b51c22cf9e1b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:54.891742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a887d7-b9b5-44e4-875e-b51c22cf9e1b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:54.897257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16463-11f4-44f5-852a-b159bc605164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:54.897915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16463-11f4-44f5-852a-b159bc605164 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:54.898203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16463-11f4-44f5-852a-b159bc605164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:54.898485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16463-11f4-44f5-852a-b159bc605164 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:54.899191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16463-11f4-44f5-852a-b159bc605164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:54.901163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16463-11f4-44f5-852a-b159bc605164 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:54.937429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1183/2362] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:10:54 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:10:54.941925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97f057-8bb5-4ddf-a07d-c6aad469fdab admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:54.952228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97f057-8bb5-4ddf-a07d-c6aad469fdab admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:55.027447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6ea15-00cf-4687-ab51-420bbc201500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:55.028115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ea15-00cf-4687-ab51-420bbc201500 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:55.028399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ea15-00cf-4687-ab51-420bbc201500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:55.028674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ea15-00cf-4687-ab51-420bbc201500 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:55.029406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ea15-00cf-4687-ab51-420bbc201500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:55.031320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ea15-00cf-4687-ab51-420bbc201500 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:55.051710 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1180/2363] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:10:54 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:10:55.055922 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16463-11f4-44f5-852a-b159bc605164 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:55.066757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16463-11f4-44f5-852a-b159bc605164 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:55.071544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a4ea0-09f6-462f-8ba3-807a97a5c325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:55.072203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a4ea0-09f6-462f-8ba3-807a97a5c325 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:55.072488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a4ea0-09f6-462f-8ba3-807a97a5c325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:55.072770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a4ea0-09f6-462f-8ba3-807a97a5c325 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:55.073538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a4ea0-09f6-462f-8ba3-807a97a5c325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:55.075118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a4ea0-09f6-462f-8ba3-807a97a5c325 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:55.085719 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1184/2364] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:10:55.089826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6ea15-00cf-4687-ab51-420bbc201500 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:55.100720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6ea15-00cf-4687-ab51-420bbc201500 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:55.105405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cd245-88fe-4a9a-8216-8ff591b08106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:55.106063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cd245-88fe-4a9a-8216-8ff591b08106 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:55.106351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cd245-88fe-4a9a-8216-8ff591b08106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:55.106633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cd245-88fe-4a9a-8216-8ff591b08106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:55.107361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cd245-88fe-4a9a-8216-8ff591b08106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:55.108959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cd245-88fe-4a9a-8216-8ff591b08106 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:55.119412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1181/2365] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:10:55.123599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a4ea0-09f6-462f-8ba3-807a97a5c325 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:55.133727 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a4ea0-09f6-462f-8ba3-807a97a5c325 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:55.138343 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa96777-d003-48d3-9ee2-796eef5ee513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:55.139209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa96777-d003-48d3-9ee2-796eef5ee513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:55.139491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa96777-d003-48d3-9ee2-796eef5ee513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:55.139765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa96777-d003-48d3-9ee2-796eef5ee513 admin admin] PATH_INFO: `/v3/projects/d197f69e1d4b49d0801e6b625599aad8/users/25945373ee26489e8014f07ec27d81f7/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:55.154898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa96777-d003-48d3-9ee2-796eef5ee513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d197f69e1d4b49d0801e6b625599aad8, user_id=25945373ee26489e8014f07ec27d81f7, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:55.157584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa96777-d003-48d3-9ee2-796eef5ee513 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:55.170683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1185/2366] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:10:55 2022] PUT /identity/v3/projects/d197f69e1d4b49d0801e6b625599aad8/users/25945373ee26489e8014f07ec27d81f7/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:10:55.176690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad0ec5-3b81-4f4b-a3fe-0d9eabb8c99b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:55.176984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad0ec5-3b81-4f4b-a3fe-0d9eabb8c99b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:55.177309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad0ec5-3b81-4f4b-a3fe-0d9eabb8c99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:55.198396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-90ad0ec5-3b81-4f4b-a3fe-0d9eabb8c99b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:10:55.199630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-90ad0ec5-3b81-4f4b-a3fe-0d9eabb8c99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:55.262557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1182/2367] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:10:55.273568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa96777-d003-48d3-9ee2-796eef5ee513 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:55.275457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3aa96777-d003-48d3-9ee2-796eef5ee513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:55.334839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa96777-d003-48d3-9ee2-796eef5ee513 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:55.339151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02e384-e8fc-48a4-a1de-8da74071b3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:55.339795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02e384-e8fc-48a4-a1de-8da74071b3ca service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:55.340094 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02e384-e8fc-48a4-a1de-8da74071b3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:55.340353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02e384-e8fc-48a4-a1de-8da74071b3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:55.350284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02e384-e8fc-48a4-a1de-8da74071b3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:55.352103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02e384-e8fc-48a4-a1de-8da74071b3ca service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:55.362621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1186/2368] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:10:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:10:59.543742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8f422-7eed-43a9-bdee-eb774efba57c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:59.544249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8f422-7eed-43a9-bdee-eb774efba57c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:59.544361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8f422-7eed-43a9-bdee-eb774efba57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:59.581888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-50e8f422-7eed-43a9-bdee-eb774efba57c None None] MFA Rules not processed for user `25945373ee26489e8014f07ec27d81f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:10:59.583893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-50e8f422-7eed-43a9-bdee-eb774efba57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:59.661269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:10:59.667205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1183/2369] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:10:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:10:59.682707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02e384-e8fc-48a4-a1de-8da74071b3ca service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:10:59.684413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6e02e384-e8fc-48a4-a1de-8da74071b3ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:10:59.763683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02e384-e8fc-48a4-a1de-8da74071b3ca service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:10:59.769318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9175e-14dc-4af8-ac61-771c5c4307a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:10:59.769921 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9175e-14dc-4af8-ac61-771c5c4307a7 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:10:59.770209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9175e-14dc-4af8-ac61-771c5c4307a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:10:59.770497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9175e-14dc-4af8-ac61-771c5c4307a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:10:59.777501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:10:59.780057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9175e-14dc-4af8-ac61-771c5c4307a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:10:59.782388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9175e-14dc-4af8-ac61-771c5c4307a7 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:10:59.794356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1187/2370] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:10:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:11:01.195767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ca733-1c5c-4229-846e-c5cf634168fa None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:11:01.195767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ca733-1c5c-4229-846e-c5cf634168fa None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:11:01.195767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ca733-1c5c-4229-846e-c5cf634168fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:11:01.217559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-230ca733-1c5c-4229-846e-c5cf634168fa None None] MFA Rules not processed for user `25945373ee26489e8014f07ec27d81f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:11:01.219118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-230ca733-1c5c-4229-846e-c5cf634168fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:11:01.227846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1184/2371] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:11:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:11:06.323850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eec859-c87a-4d09-a483-c9c1bb771c04 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:11:06.324405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eec859-c87a-4d09-a483-c9c1bb771c04 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:11:06.324405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eec859-c87a-4d09-a483-c9c1bb771c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:11:06.353255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a8eec859-c87a-4d09-a483-c9c1bb771c04 None None] MFA Rules not processed for user `2b973f8f63eb49f19895a43313f2bc55`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:11:06.354317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a8eec859-c87a-4d09-a483-c9c1bb771c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:11:06.412872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1188/2372] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:11:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:11:06.725435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ca733-1c5c-4229-846e-c5cf634168fa None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:11:06.735265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ca733-1c5c-4229-846e-c5cf634168fa None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:11:06.739574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1283d4-e3c6-44bf-acc9-9bb7030efa5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:11:06.740226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1283d4-e3c6-44bf-acc9-9bb7030efa5d service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:11:06.740516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1283d4-e3c6-44bf-acc9-9bb7030efa5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:11:06.740797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1283d4-e3c6-44bf-acc9-9bb7030efa5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:11:06.742643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-de1283d4-e3c6-44bf-acc9-9bb7030efa5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:11:06.753212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1283d4-e3c6-44bf-acc9-9bb7030efa5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:11:06.754668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1283d4-e3c6-44bf-acc9-9bb7030efa5d service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:11:06.762981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1185/2373] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:11:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:11:16.403946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=27 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:11:16.759384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=10 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:11:37.539857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eec859-c87a-4d09-a483-c9c1bb771c04 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:11:37.541300 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a8eec859-c87a-4d09-a483-c9c1bb771c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:11:37.599168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eec859-c87a-4d09-a483-c9c1bb771c04 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:11:37.602760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84ecb0-b6a1-45b6-a693-14559eb8e94e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:11:37.603403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84ecb0-b6a1-45b6-a693-14559eb8e94e service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:11:37.603691 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84ecb0-b6a1-45b6-a693-14559eb8e94e service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:11:37.603964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84ecb0-b6a1-45b6-a693-14559eb8e94e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:11:37.605751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7f84ecb0-b6a1-45b6-a693-14559eb8e94e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:11:37.671480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84ecb0-b6a1-45b6-a693-14559eb8e94e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:11:37.672929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84ecb0-b6a1-45b6-a693-14559eb8e94e service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:11:37.682878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1189/2374] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:11:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:11:47.678614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=48 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:11:49.726332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ad836-01c1-406b-896b-9b7dfbd3a1ac None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:11:49.726864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ad836-01c1-406b-896b-9b7dfbd3a1ac None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:11:49.726991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ad836-01c1-406b-896b-9b7dfbd3a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:11:49.744565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6c0ad836-01c1-406b-896b-9b7dfbd3a1ac None None] MFA Rules not processed for user `569256e6d1d1418eafb5ff2ed8909926`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:11:49.745815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6c0ad836-01c1-406b-896b-9b7dfbd3a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:11:49.753028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1186/2375] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:11:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:11:59.739425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:12:08.341223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84ecb0-b6a1-45b6-a693-14559eb8e94e service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:08.351340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84ecb0-b6a1-45b6-a693-14559eb8e94e service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:08.354989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabd93d-b3ac-4787-886a-5b4f45210d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:08.355629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd93d-b3ac-4787-886a-5b4f45210d63 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:08.355915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd93d-b3ac-4787-886a-5b4f45210d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:08.356196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd93d-b3ac-4787-886a-5b4f45210d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:08.358018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3fabd93d-b3ac-4787-886a-5b4f45210d63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:08.425346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabd93d-b3ac-4787-886a-5b4f45210d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:08.427259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabd93d-b3ac-4787-886a-5b4f45210d63 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:08.437721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1190/2376] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:12:18.433212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=30 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:12:25.541217 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ad836-01c1-406b-896b-9b7dfbd3a1ac None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:25.542913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6c0ad836-01c1-406b-896b-9b7dfbd3a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:25.555300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ad836-01c1-406b-896b-9b7dfbd3a1ac None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:25.559018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ef800-9e51-4039-b7bf-d8fb799eb2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:25.559680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ef800-9e51-4039-b7bf-d8fb799eb2ef service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:25.559973 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ef800-9e51-4039-b7bf-d8fb799eb2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:25.560251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ef800-9e51-4039-b7bf-d8fb799eb2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:25.562078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-071ef800-9e51-4039-b7bf-d8fb799eb2ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:25.623257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ef800-9e51-4039-b7bf-d8fb799eb2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:25.624720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ef800-9e51-4039-b7bf-d8fb799eb2ef service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:25.634418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1187/2377] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:12:26.647818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabd93d-b3ac-4787-886a-5b4f45210d63 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:26.661051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabd93d-b3ac-4787-886a-5b4f45210d63 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:26.668016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90bc7d-c15a-4b97-bc65-0e206ae205fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:26.668776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90bc7d-c15a-4b97-bc65-0e206ae205fe service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:26.669063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90bc7d-c15a-4b97-bc65-0e206ae205fe service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:26.669451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90bc7d-c15a-4b97-bc65-0e206ae205fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:26.672102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ba90bc7d-c15a-4b97-bc65-0e206ae205fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:26.685238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90bc7d-c15a-4b97-bc65-0e206ae205fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:26.686787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90bc7d-c15a-4b97-bc65-0e206ae205fe service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:26.699058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1191/2378] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:12:31.942827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ef800-9e51-4039-b7bf-d8fb799eb2ef service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:31.951875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ef800-9e51-4039-b7bf-d8fb799eb2ef service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:31.955682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7bb96-6f57-47bc-8645-a9a406931d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:31.956350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7bb96-6f57-47bc-8645-a9a406931d46 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:31.956637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7bb96-6f57-47bc-8645-a9a406931d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:31.956922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7bb96-6f57-47bc-8645-a9a406931d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:31.959435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cfa7bb96-6f57-47bc-8645-a9a406931d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:31.972801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7bb96-6f57-47bc-8645-a9a406931d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:31.974297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7bb96-6f57-47bc-8645-a9a406931d46 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:31.989649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1188/2379] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:12:35.235843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2c5bd-8f7e-4993-b746-569278ddf12a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:35.236440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2c5bd-8f7e-4993-b746-569278ddf12a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:35.236440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2c5bd-8f7e-4993-b746-569278ddf12a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:35.240170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1192/2380] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jun 21 19:12:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 19:12:35.248853 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19541d76-5587-49e8-997a-c294b4f4a78b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:35.249379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19541d76-5587-49e8-997a-c294b4f4a78b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:35.249480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19541d76-5587-49e8-997a-c294b4f4a78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:35.323841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-19541d76-5587-49e8-997a-c294b4f4a78b None None] MFA Rules not processed for user `f0e3fc619dc74a0591d8ad14369ae673`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:12:35.323841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-19541d76-5587-49e8-997a-c294b4f4a78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:35.424036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1189/2381] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jun 21 19:12:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:12:35.435458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2c5bd-8f7e-4993-b746-569278ddf12a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:35.437336 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-05b2c5bd-8f7e-4993-b746-569278ddf12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:35.456585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2c5bd-8f7e-4993-b746-569278ddf12a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:35.461890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff536c4e-1976-40f0-bfe3-5fd0ac984cc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:35.462635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff536c4e-1976-40f0-bfe3-5fd0ac984cc3 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:35.462988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff536c4e-1976-40f0-bfe3-5fd0ac984cc3 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:35.463337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff536c4e-1976-40f0-bfe3-5fd0ac984cc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:35.472715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff536c4e-1976-40f0-bfe3-5fd0ac984cc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:35.474506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff536c4e-1976-40f0-bfe3-5fd0ac984cc3 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:35.485281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1193/2382] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:12:38.001810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19541d76-5587-49e8-997a-c294b4f4a78b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:38.016066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19541d76-5587-49e8-997a-c294b4f4a78b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:38.021032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e713787-854e-4da3-8332-e5f14d26991b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:38.021758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e713787-854e-4da3-8332-e5f14d26991b service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:38.022063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e713787-854e-4da3-8332-e5f14d26991b service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:38.022446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e713787-854e-4da3-8332-e5f14d26991b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:38.034795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1e713787-854e-4da3-8332-e5f14d26991b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:38.052500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e713787-854e-4da3-8332-e5f14d26991b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:38.054005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e713787-854e-4da3-8332-e5f14d26991b service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:38.067683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1190/2383] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:12:38.834356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff536c4e-1976-40f0-bfe3-5fd0ac984cc3 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:38.836779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ff536c4e-1976-40f0-bfe3-5fd0ac984cc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:38.856853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff536c4e-1976-40f0-bfe3-5fd0ac984cc3 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:38.861571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:38.862476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:38.862758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:38.863043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] PATH_INFO: `/v3/users/c19aa025ff3046229cf0b8041058285e` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:38.864293 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c19aa025ff3046229cf0b8041058285e)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:38.866277 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:38.919303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c19aa025ff3046229cf0b8041058285e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:38.931435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c19aa025ff3046229cf0b8041058285e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:38.938877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c19aa025ff3046229cf0b8041058285e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:38.939503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c19aa025ff3046229cf0b8041058285e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:38.947068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c19aa025ff3046229cf0b8041058285e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:38.958257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1194/2384] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:12:38 2022] DELETE /identity/v3/users/c19aa025ff3046229cf0b8041058285e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:12:39.149898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c240b5-278e-4d74-a11b-8efffedbea7c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:39.150533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c240b5-278e-4d74-a11b-8efffedbea7c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:39.150533 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c240b5-278e-4d74-a11b-8efffedbea7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:39.161416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:39.162942 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:39.170399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-13c240b5-278e-4d74-a11b-8efffedbea7c None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:12:39.171633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-13c240b5-278e-4d74-a11b-8efffedbea7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:39.226410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:12:39.236347 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1191/2385] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:12:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:12:39.247145 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1bdaf-dd68-4993-990d-ecb38fc648e4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:39.252373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:39.253265 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:39.253544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:39.253815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] PATH_INFO: `/v3/projects/4202625ef50b44de9a96563feb874a5a` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:39.255139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4202625ef50b44de9a96563feb874a5a)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:39.257137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:39.301584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c240b5-278e-4d74-a11b-8efffedbea7c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:39.303392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-13c240b5-278e-4d74-a11b-8efffedbea7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:39.333953 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:12:39.335709 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4202625ef50b44de9a96563feb874a5a', 'request_id': 'req-d430495e-95d8-4014-92c8-e56184b62d7f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.343878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4202625ef50b44de9a96563feb874a5a', 'request_id': 'req-d430495e-95d8-4014-92c8-e56184b62d7f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.359312 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4202625ef50b44de9a96563feb874a5a', 'request_id': 'req-d430495e-95d8-4014-92c8-e56184b62d7f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.374087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] The token cache is being invalidate because project 4202625ef50b44de9a96563feb874a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:12:39.374613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.375472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.379262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1195/2386] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:12:39 2022] DELETE /identity/v3/projects/4202625ef50b44de9a96563feb874a5a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:12:39.383924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:39.386890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:39.417275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c240b5-278e-4d74-a11b-8efffedbea7c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:39.474888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430495e-95d8-4014-92c8-e56184b62d7f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:39.480195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:39.481073 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:39.481388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:39.481671 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] PATH_INFO: `/v3/users/4468571e5cb1492eb10c7cd61e0b6a19` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:39.482936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4468571e5cb1492eb10c7cd61e0b6a19)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:39.484885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:39.488287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1083c-b267-41e2-afa1-cc375db401df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:39.488948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1083c-b267-41e2-afa1-cc375db401df service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:39.489270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1083c-b267-41e2-afa1-cc375db401df service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:39.489625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1083c-b267-41e2-afa1-cc375db401df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:39.491440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-18b1083c-b267-41e2-afa1-cc375db401df service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:39.536200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4468571e5cb1492eb10c7cd61e0b6a19'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.549609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4468571e5cb1492eb10c7cd61e0b6a19'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.556632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1083c-b267-41e2-afa1-cc375db401df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:39.558314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1083c-b267-41e2-afa1-cc375db401df service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:39.562533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4468571e5cb1492eb10c7cd61e0b6a19'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.563390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4468571e5cb1492eb10c7cd61e0b6a19'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.568977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1192/2387] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:12:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 19:12:39.571461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4468571e5cb1492eb10c7cd61e0b6a19'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:39.577883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1196/2388] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:12:39 2022] DELETE /identity/v3/users/4468571e5cb1492eb10c7cd61e0b6a19 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:12:39.828457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1083c-b267-41e2-afa1-cc375db401df service swift] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:39.840037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1083c-b267-41e2-afa1-cc375db401df service swift] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:39.910040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:12:39.936914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:39.938204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:39.938662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:39.938945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] PATH_INFO: `/v3/projects/5f61dcd16eac4e22bb135b168d015d4d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:39.940725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f61dcd16eac4e22bb135b168d015d4d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:39.942871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:40.007166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f61dcd16eac4e22bb135b168d015d4d', 'request_id': 'req-afcd0950-b212-4329-9580-eb6eab63edbb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:40.016177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f61dcd16eac4e22bb135b168d015d4d', 'request_id': 'req-afcd0950-b212-4329-9580-eb6eab63edbb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:40.032823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f61dcd16eac4e22bb135b168d015d4d', 'request_id': 'req-afcd0950-b212-4329-9580-eb6eab63edbb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:40.052192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] The token cache is being invalidate because project 5f61dcd16eac4e22bb135b168d015d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:12:40.052619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:40.053784 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:12:40.057127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1193/2389] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:12:39 2022] DELETE /identity/v3/projects/5f61dcd16eac4e22bb135b168d015d4d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:12:41.797482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:41.801187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:41.886007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebb412-9acc-457b-894e-9f9124122dc8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:41.897718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:12:41.965781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:41.966539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:41.966712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:41.966902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:41.968900 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:42.091415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:42.093105 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:42.105765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1197/2390] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:12:50.047783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:12:51.725385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:51.726866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:51.743278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd0950-b212-4329-9580-eb6eab63edbb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:51.748883 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:51.749581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:51.749870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:51.750153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:51.752589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:51.832722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:51.834370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:51.843395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1194/2391] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:12:52.100089 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=49 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:12:54.022824 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:54.025312 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:54.085062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430614c-93a4-44d9-af3f-0415974384bd service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:54.088751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915704f26c814eaa9e6c2a68533a9c66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:54.089531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:54.089697 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:54.089973 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:54.091818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:54.106078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:54.107526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:54.118937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1198/2392] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:12:54.184216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:54.185795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:54.198290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649daab-9d0e-4d69-b9f7-8cff0caa2148 service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:54.201909 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9171be-c182-4440-bfcf-efc9f9c86f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:54.202547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9171be-c182-4440-bfcf-efc9f9c86f78 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:54.202836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9171be-c182-4440-bfcf-efc9f9c86f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:54.203117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9171be-c182-4440-bfcf-efc9f9c86f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:54.205442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2a9171be-c182-4440-bfcf-efc9f9c86f78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:54.219161 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9171be-c182-4440-bfcf-efc9f9c86f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:54.220649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9171be-c182-4440-bfcf-efc9f9c86f78 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:54.229237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1195/2393] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:12:54.302535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:54.304014 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:54.316483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d71959-64d6-45b0-aede-082451d1c423 service glance] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:54.320818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:54.321325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:54.321717 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:54.321886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:54.323813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:54.339092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:54.340526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:54.352148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1199/2394] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:12:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:12:55.976456 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769df537-2677-4989-9615-20991c2695f8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:55.976975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769df537-2677-4989-9615-20991c2695f8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:55.977078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769df537-2677-4989-9615-20991c2695f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:55.994872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-769df537-2677-4989-9615-20991c2695f8 None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:12:55.996076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-769df537-2677-4989-9615-20991c2695f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:56.050955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1196/2395] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:12:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:12:56.062524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:12:56.063944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:12:56.133898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58670bc-79ce-4579-8a7b-f3f07fe5b628 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:12:56.139095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81506ed7-bdff-4668-807b-f5ada434eb3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:12:56.139656 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81506ed7-bdff-4668-807b-f5ada434eb3d service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:12:56.139944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81506ed7-bdff-4668-807b-f5ada434eb3d service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:12:56.140220 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81506ed7-bdff-4668-807b-f5ada434eb3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:12:56.149141 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81506ed7-bdff-4668-807b-f5ada434eb3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:12:56.150730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81506ed7-bdff-4668-807b-f5ada434eb3d service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:12:56.159081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1200/2396] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:12:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:13:06.043754 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=68 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:13:06.155578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=64 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:13:18.958545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769df537-2677-4989-9615-20991c2695f8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:18.967993 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769df537-2677-4989-9615-20991c2695f8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:18.971613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20996da2-8b6b-4b3c-a679-b45bac277a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:18.972272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20996da2-8b6b-4b3c-a679-b45bac277a07 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:18.972556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20996da2-8b6b-4b3c-a679-b45bac277a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:18.972835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20996da2-8b6b-4b3c-a679-b45bac277a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:18.974749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-20996da2-8b6b-4b3c-a679-b45bac277a07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:18.984651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20996da2-8b6b-4b3c-a679-b45bac277a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:18.986139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20996da2-8b6b-4b3c-a679-b45bac277a07 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:18.994392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1197/2397] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:13:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:13:21.385398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4860a-b0df-440e-b0e0-2e871374540b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:21.385896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4860a-b0df-440e-b0e0-2e871374540b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:21.386007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4860a-b0df-440e-b0e0-2e871374540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:21.405320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-50a4860a-b0df-440e-b0e0-2e871374540b None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:13:21.406529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-50a4860a-b0df-440e-b0e0-2e871374540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:21.414501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1201/2398] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jun 21 19:13:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:13:21.427248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20996da2-8b6b-4b3c-a679-b45bac277a07 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:21.439381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20996da2-8b6b-4b3c-a679-b45bac277a07 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:21.444622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a6a18-297c-4cbe-af42-f854d2bbd377 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:21.445290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a6a18-297c-4cbe-af42-f854d2bbd377 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:21.445575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a6a18-297c-4cbe-af42-f854d2bbd377 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:21.445852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a6a18-297c-4cbe-af42-f854d2bbd377 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:21.454897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a6a18-297c-4cbe-af42-f854d2bbd377 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:21.456480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a6a18-297c-4cbe-af42-f854d2bbd377 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:21.465341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1198/2399] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:13:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:13:30.479136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976cdac-ba08-4d63-bbfc-101e06769e0d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:30.479642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976cdac-ba08-4d63-bbfc-101e06769e0d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:30.479760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976cdac-ba08-4d63-bbfc-101e06769e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:30.499149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5976cdac-ba08-4d63-bbfc-101e06769e0d None None] MFA Rules not processed for user `25945373ee26489e8014f07ec27d81f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:13:30.500327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5976cdac-ba08-4d63-bbfc-101e06769e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:30.553479 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1202/2400] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:13:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:13:31.462066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=12 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:13:40.546776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=28 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:13:44.480242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a6a18-297c-4cbe-af42-f854d2bbd377 service swift] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:44.492864 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a6a18-297c-4cbe-af42-f854d2bbd377 service swift] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:44.498309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:44.498948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:44.499231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:44.499537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:44.501358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:44.516163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:44.517928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:44.529618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1199/2401] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:13:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:13:51.935146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976cdac-ba08-4d63-bbfc-101e06769e0d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:51.944954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976cdac-ba08-4d63-bbfc-101e06769e0d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:51.948690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7378826-895b-436f-8de0-425e12ba18dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:51.949466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7378826-895b-436f-8de0-425e12ba18dc service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:51.949737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7378826-895b-436f-8de0-425e12ba18dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:51.950016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7378826-895b-436f-8de0-425e12ba18dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:51.951778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a7378826-895b-436f-8de0-425e12ba18dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:52.016155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7378826-895b-436f-8de0-425e12ba18dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:52.017667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7378826-895b-436f-8de0-425e12ba18dc service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:52.028505 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1203/2402] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:13:54.524566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:13:55.109906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:55.111517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:55.126164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ab297-082a-48e7-9d63-e9267f3982d6 service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:55.130839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:55.131725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.132010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.132857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] PATH_INFO: `/v3/users/2b973f8f63eb49f19895a43313f2bc55` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.148821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b973f8f63eb49f19895a43313f2bc55)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:55.150828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:55.203120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b973f8f63eb49f19895a43313f2bc55'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.215406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b973f8f63eb49f19895a43313f2bc55'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.222214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b973f8f63eb49f19895a43313f2bc55'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.229752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b973f8f63eb49f19895a43313f2bc55'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.230399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d34e379e-b64b-47d8-834b-1670b774a002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b973f8f63eb49f19895a43313f2bc55'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.237947 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1200/2403] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:13:55 2022] DELETE /identity/v3/users/2b973f8f63eb49f19895a43313f2bc55 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:13:55.413898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7378826-895b-436f-8de0-425e12ba18dc service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:55.424330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7378826-895b-436f-8de0-425e12ba18dc service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:55.483034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:55.483908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.484192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.484475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] PATH_INFO: `/v3/projects/b6bbb7a5373e46f69acab406d82d87f3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.495266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6bbb7a5373e46f69acab406d82d87f3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:55.497206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:55.543626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6bbb7a5373e46f69acab406d82d87f3', 'request_id': 'req-a7f077e5-a77e-4433-ab5f-b74af452ac9b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.550171 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6bbb7a5373e46f69acab406d82d87f3', 'request_id': 'req-a7f077e5-a77e-4433-ab5f-b74af452ac9b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.563002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6bbb7a5373e46f69acab406d82d87f3', 'request_id': 'req-a7f077e5-a77e-4433-ab5f-b74af452ac9b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.574728 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] The token cache is being invalidate because project b6bbb7a5373e46f69acab406d82d87f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:13:55.575037 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.575791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a7f077e5-a77e-4433-ab5f-b74af452ac9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:13:55.577464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1204/2404] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:13:55 2022] DELETE /identity/v3/projects/b6bbb7a5373e46f69acab406d82d87f3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:13:55.582369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e788b-13d1-48bd-bb50-a4cb4a6f5005 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.582875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e788b-13d1-48bd-bb50-a4cb4a6f5005 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.582987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e788b-13d1-48bd-bb50-a4cb4a6f5005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.602538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-778e788b-13d1-48bd-bb50-a4cb4a6f5005 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:13:55.603787 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-778e788b-13d1-48bd-bb50-a4cb4a6f5005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:55.664204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1201/2405] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:13:55.672161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a89d0f-fe66-4bc7-8d6f-7e3ea25136df None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.672544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a89d0f-fe66-4bc7-8d6f-7e3ea25136df None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.672879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a89d0f-fe66-4bc7-8d6f-7e3ea25136df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.691559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-89a89d0f-fe66-4bc7-8d6f-7e3ea25136df None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:13:55.693030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-89a89d0f-fe66-4bc7-8d6f-7e3ea25136df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:55.701729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1205/2406] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:13:55.706016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e788b-13d1-48bd-bb50-a4cb4a6f5005 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:55.715770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e788b-13d1-48bd-bb50-a4cb4a6f5005 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:55.720388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587b1b9-f8e5-424d-a746-fc5d4965f1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:55.721036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587b1b9-f8e5-424d-a746-fc5d4965f1eb admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.721348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587b1b9-f8e5-424d-a746-fc5d4965f1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.721632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587b1b9-f8e5-424d-a746-fc5d4965f1eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.722364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587b1b9-f8e5-424d-a746-fc5d4965f1eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:13:55.722676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587b1b9-f8e5-424d-a746-fc5d4965f1eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:55.724272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587b1b9-f8e5-424d-a746-fc5d4965f1eb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:55.735920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1202/2407] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:13:55.740219 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a89d0f-fe66-4bc7-8d6f-7e3ea25136df None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:55.749972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a89d0f-fe66-4bc7-8d6f-7e3ea25136df None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:55.754676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fb805-f18d-454d-a35b-c6170f586531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:55.755317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fb805-f18d-454d-a35b-c6170f586531 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.755603 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fb805-f18d-454d-a35b-c6170f586531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.755905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fb805-f18d-454d-a35b-c6170f586531 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.756588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fb805-f18d-454d-a35b-c6170f586531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:55.759098 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fb805-f18d-454d-a35b-c6170f586531 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:55.763162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:13:55.786279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1206/2408] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:13:55 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:13:55.790687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587b1b9-f8e5-424d-a746-fc5d4965f1eb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:55.801692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587b1b9-f8e5-424d-a746-fc5d4965f1eb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:55.846564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:13:55.860619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af42932-d07d-413f-a018-bf61200f42ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:55.861311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af42932-d07d-413f-a018-bf61200f42ad admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.861594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af42932-d07d-413f-a018-bf61200f42ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.861868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af42932-d07d-413f-a018-bf61200f42ad admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.862572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af42932-d07d-413f-a018-bf61200f42ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:55.864512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af42932-d07d-413f-a018-bf61200f42ad admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:55.884169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1203/2409] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:13:55 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:13:55.888286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fb805-f18d-454d-a35b-c6170f586531 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:55.898750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fb805-f18d-454d-a35b-c6170f586531 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:55.903519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1698e-b419-41ce-be41-47d507cf8f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:55.904185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1698e-b419-41ce-be41-47d507cf8f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.904496 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1698e-b419-41ce-be41-47d507cf8f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.904776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1698e-b419-41ce-be41-47d507cf8f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.905559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1698e-b419-41ce-be41-47d507cf8f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:55.907137 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1698e-b419-41ce-be41-47d507cf8f3e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:55.917758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1207/2410] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:13:55.921547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af42932-d07d-413f-a018-bf61200f42ad admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:55.931831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af42932-d07d-413f-a018-bf61200f42ad admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:55.936590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d2563-40aa-40c9-94fc-4c7d6d7f9647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:55.937289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d2563-40aa-40c9-94fc-4c7d6d7f9647 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.937581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d2563-40aa-40c9-94fc-4c7d6d7f9647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.937862 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d2563-40aa-40c9-94fc-4c7d6d7f9647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.938648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d2563-40aa-40c9-94fc-4c7d6d7f9647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:55.940324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d2563-40aa-40c9-94fc-4c7d6d7f9647 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:55.950703 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1204/2411] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:13:55.954841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1698e-b419-41ce-be41-47d507cf8f3e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:55.964804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1698e-b419-41ce-be41-47d507cf8f3e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:55.969396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9025d-635c-4118-86bb-907a326a126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:55.970265 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9025d-635c-4118-86bb-907a326a126f admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:55.970548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9025d-635c-4118-86bb-907a326a126f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:55.970828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9025d-635c-4118-86bb-907a326a126f admin admin] PATH_INFO: `/v3/projects/6cfb06fcf112450ba272a9254822bf56/users/c908bcb3869442b2bd8c630ae1e5d9c1/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:55.985714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9025d-635c-4118-86bb-907a326a126f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfb06fcf112450ba272a9254822bf56, user_id=c908bcb3869442b2bd8c630ae1e5d9c1, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:55.988346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9025d-635c-4118-86bb-907a326a126f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:56.001919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1208/2412] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:13:55 2022] PUT /identity/v3/projects/6cfb06fcf112450ba272a9254822bf56/users/c908bcb3869442b2bd8c630ae1e5d9c1/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:13:56.012512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be157c46-1a57-4da4-9c87-a8710ad993ef None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:56.012803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be157c46-1a57-4da4-9c87-a8710ad993ef None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:56.013103 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be157c46-1a57-4da4-9c87-a8710ad993ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:56.040336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-be157c46-1a57-4da4-9c87-a8710ad993ef None None] MFA Rules not processed for user `c908bcb3869442b2bd8c630ae1e5d9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:13:56.041593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-be157c46-1a57-4da4-9c87-a8710ad993ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:56.106739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1205/2413] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:13:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:13:56.118585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9025d-635c-4118-86bb-907a326a126f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:13:56.120301 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c6d9025d-635c-4118-86bb-907a326a126f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:13:56.197210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9025d-635c-4118-86bb-907a326a126f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:13:56.202799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859efc4-57f2-4cb7-bc0b-d13eaa32fe38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:13:56.203442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859efc4-57f2-4cb7-bc0b-d13eaa32fe38 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:13:56.203770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859efc4-57f2-4cb7-bc0b-d13eaa32fe38 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:13:56.204041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859efc4-57f2-4cb7-bc0b-d13eaa32fe38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:13:56.211860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859efc4-57f2-4cb7-bc0b-d13eaa32fe38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:13:56.213501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859efc4-57f2-4cb7-bc0b-d13eaa32fe38 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:13:56.222362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1209/2414] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:13:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:05.569209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be157c46-1a57-4da4-9c87-a8710ad993ef None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:05.629337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be157c46-1a57-4da4-9c87-a8710ad993ef None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:05.633993 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:05.634944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:05.635234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:05.635625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] PATH_INFO: `/v3/users/c908bcb3869442b2bd8c630ae1e5d9c1` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:05.636827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c908bcb3869442b2bd8c630ae1e5d9c1)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:05.638803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:05.684475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c908bcb3869442b2bd8c630ae1e5d9c1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:05.692375 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:05.697265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c908bcb3869442b2bd8c630ae1e5d9c1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:05.705008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c908bcb3869442b2bd8c630ae1e5d9c1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:05.712116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c908bcb3869442b2bd8c630ae1e5d9c1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:05.712809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c908bcb3869442b2bd8c630ae1e5d9c1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:05.718825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1206/2415] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:05 2022] DELETE /identity/v3/users/c908bcb3869442b2bd8c630ae1e5d9c1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:14:05.723297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65372d-93af-4747-868c-dce7ee38c6b0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:05.723813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65372d-93af-4747-868c-dce7ee38c6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:05.723936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65372d-93af-4747-868c-dce7ee38c6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:05.741039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5f65372d-93af-4747-868c-dce7ee38c6b0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:05.742888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5f65372d-93af-4747-868c-dce7ee38c6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:05.801455 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1210/2416] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:05.811670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:05.813160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:05.870140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c9cbf-d6b9-4606-ae0a-e1bbf7f6dac6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:05.873716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd09c7-e31a-4984-a28d-c21d9a068e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:05.874353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd09c7-e31a-4984-a28d-c21d9a068e53 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:05.874639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd09c7-e31a-4984-a28d-c21d9a068e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:05.874917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd09c7-e31a-4984-a28d-c21d9a068e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:05.883814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd09c7-e31a-4984-a28d-c21d9a068e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:05.885285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd09c7-e31a-4984-a28d-c21d9a068e53 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:05.894566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1207/2417] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:06.158752 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65372d-93af-4747-868c-dce7ee38c6b0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:06.160275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5f65372d-93af-4747-868c-dce7ee38c6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:06.173413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65372d-93af-4747-868c-dce7ee38c6b0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:06.178228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:06.179095 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.179382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.179663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] PATH_INFO: `/v3/projects/6cfb06fcf112450ba272a9254822bf56` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.180871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfb06fcf112450ba272a9254822bf56)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:06.182893 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:06.218976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:06.233730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfb06fcf112450ba272a9254822bf56', 'request_id': 'req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:06.240395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfb06fcf112450ba272a9254822bf56', 'request_id': 'req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:06.247941 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfb06fcf112450ba272a9254822bf56', 'request_id': 'req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:06.258513 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] The token cache is being invalidate because project 6cfb06fcf112450ba272a9254822bf56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:06.258804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:06.259427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d9b227e1-89e8-4b83-9935-ff8c8c8f9c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:06.260971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1211/2418] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:06 2022] DELETE /identity/v3/projects/6cfb06fcf112450ba272a9254822bf56 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:06.265932 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24de80-3aed-4bf5-b0a7-40748d3e07d0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.266254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24de80-3aed-4bf5-b0a7-40748d3e07d0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.266538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24de80-3aed-4bf5-b0a7-40748d3e07d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.284931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9a24de80-3aed-4bf5-b0a7-40748d3e07d0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:06.286217 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9a24de80-3aed-4bf5-b0a7-40748d3e07d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:06.349983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1208/2419] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:06.357614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9d69d-a671-4596-98df-6da06d2e6a85 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.357909 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9d69d-a671-4596-98df-6da06d2e6a85 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.358194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9d69d-a671-4596-98df-6da06d2e6a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.377549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b6b9d69d-a671-4596-98df-6da06d2e6a85 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:06.378754 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b6b9d69d-a671-4596-98df-6da06d2e6a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:06.387306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1212/2420] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:06.391614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24de80-3aed-4bf5-b0a7-40748d3e07d0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:06.402099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24de80-3aed-4bf5-b0a7-40748d3e07d0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:06.406683 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2514093-0548-48d5-bc9c-caefac8f11c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:06.407334 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2514093-0548-48d5-bc9c-caefac8f11c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.407628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2514093-0548-48d5-bc9c-caefac8f11c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.408025 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2514093-0548-48d5-bc9c-caefac8f11c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.408767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2514093-0548-48d5-bc9c-caefac8f11c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:14:06.409139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2514093-0548-48d5-bc9c-caefac8f11c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:06.410735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2514093-0548-48d5-bc9c-caefac8f11c8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:06.422318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1209/2421] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:14:06.427213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9d69d-a671-4596-98df-6da06d2e6a85 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:06.437236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9d69d-a671-4596-98df-6da06d2e6a85 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:06.441920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437653a-f28e-4195-9610-ecb0a9fdb3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:06.442837 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437653a-f28e-4195-9610-ecb0a9fdb3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.443121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437653a-f28e-4195-9610-ecb0a9fdb3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.443397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437653a-f28e-4195-9610-ecb0a9fdb3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.444100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437653a-f28e-4195-9610-ecb0a9fdb3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:06.446544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437653a-f28e-4195-9610-ecb0a9fdb3a9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:06.471845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1213/2422] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:06 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:14:06.476113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2514093-0548-48d5-bc9c-caefac8f11c8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:06.486058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2514093-0548-48d5-bc9c-caefac8f11c8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:06.541166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac528b-142f-4935-8240-c5f63a1bffa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:06.541826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac528b-142f-4935-8240-c5f63a1bffa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.542106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac528b-142f-4935-8240-c5f63a1bffa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.542383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac528b-142f-4935-8240-c5f63a1bffa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.543074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac528b-142f-4935-8240-c5f63a1bffa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:06.545012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac528b-142f-4935-8240-c5f63a1bffa3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:06.554521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:06.566185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1210/2423] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:06 2022] POST /identity/v3/users => generated 278 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:06.571221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437653a-f28e-4195-9610-ecb0a9fdb3a9 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:06.581531 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437653a-f28e-4195-9610-ecb0a9fdb3a9 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:06.586757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339ca1c-0562-4d45-ae7d-ab13e270de34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:06.587411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339ca1c-0562-4d45-ae7d-ab13e270de34 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.587696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339ca1c-0562-4d45-ae7d-ab13e270de34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.587976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339ca1c-0562-4d45-ae7d-ab13e270de34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.588694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339ca1c-0562-4d45-ae7d-ab13e270de34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:06.590323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339ca1c-0562-4d45-ae7d-ab13e270de34 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:06.601183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1214/2424] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:06.605366 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac528b-142f-4935-8240-c5f63a1bffa3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:06.615594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac528b-142f-4935-8240-c5f63a1bffa3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:06.620246 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85311b2b-b3ef-418b-a188-61b7afea3327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:06.620909 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85311b2b-b3ef-418b-a188-61b7afea3327 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.621228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85311b2b-b3ef-418b-a188-61b7afea3327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.621509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85311b2b-b3ef-418b-a188-61b7afea3327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.622241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85311b2b-b3ef-418b-a188-61b7afea3327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:06.623849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85311b2b-b3ef-418b-a188-61b7afea3327 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:06.634698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1211/2425] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:14:06.638801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339ca1c-0562-4d45-ae7d-ab13e270de34 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:06.649375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339ca1c-0562-4d45-ae7d-ab13e270de34 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:06.653947 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d4ebc-f992-4f08-986e-28edbd8c62d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:06.654806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d4ebc-f992-4f08-986e-28edbd8c62d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.655086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d4ebc-f992-4f08-986e-28edbd8c62d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.655378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d4ebc-f992-4f08-986e-28edbd8c62d8 admin admin] PATH_INFO: `/v3/projects/9bc58686e6914cf9b5cc10604ca04d39/users/cbef6b12e954408abe4ee1f7ea3e898f/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.669761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d4ebc-f992-4f08-986e-28edbd8c62d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc58686e6914cf9b5cc10604ca04d39, user_id=cbef6b12e954408abe4ee1f7ea3e898f, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:06.672502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d4ebc-f992-4f08-986e-28edbd8c62d8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:06.685519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1215/2426] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:06 2022] PUT /identity/v3/projects/9bc58686e6914cf9b5cc10604ca04d39/users/cbef6b12e954408abe4ee1f7ea3e898f/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:14:06.696581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b74743-2d4b-422a-a1dc-b05d9272e32e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.696880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b74743-2d4b-422a-a1dc-b05d9272e32e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.697199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b74743-2d4b-422a-a1dc-b05d9272e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.724168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c8b74743-2d4b-422a-a1dc-b05d9272e32e None None] MFA Rules not processed for user `cbef6b12e954408abe4ee1f7ea3e898f`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:06.725429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c8b74743-2d4b-422a-a1dc-b05d9272e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:06.789169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1212/2427] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:06 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:06.793590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d4ebc-f992-4f08-986e-28edbd8c62d8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:06.854946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d4ebc-f992-4f08-986e-28edbd8c62d8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:06.859536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1fb03-d2a1-43fe-9e98-061d1cb641b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:06.860272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1fb03-d2a1-43fe-9e98-061d1cb641b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.860462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1fb03-d2a1-43fe-9e98-061d1cb641b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.860744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1fb03-d2a1-43fe-9e98-061d1cb641b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.861498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1fb03-d2a1-43fe-9e98-061d1cb641b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:06.863432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1fb03-d2a1-43fe-9e98-061d1cb641b1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:06.889515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1216/2428] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:06 2022] POST /identity/v3/projects => generated 355 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:06.893905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b74743-2d4b-422a-a1dc-b05d9272e32e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:06.904911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b74743-2d4b-422a-a1dc-b05d9272e32e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:06.962976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf69fe3-d794-46b1-ab18-0b2238a4e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:06.963630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf69fe3-d794-46b1-ab18-0b2238a4e87e admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:06.963921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf69fe3-d794-46b1-ab18-0b2238a4e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:06.964204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf69fe3-d794-46b1-ab18-0b2238a4e87e admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:06.964891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf69fe3-d794-46b1-ab18-0b2238a4e87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:06.966847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf69fe3-d794-46b1-ab18-0b2238a4e87e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:06.987584 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1213/2429] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:06 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:06.991509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1fb03-d2a1-43fe-9e98-061d1cb641b1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:07.001772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1fb03-d2a1-43fe-9e98-061d1cb641b1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:07.006896 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5301b4d-3339-4fb2-a7d2-1cd6cfaa5557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:07.007590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5301b4d-3339-4fb2-a7d2-1cd6cfaa5557 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:07.007879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5301b4d-3339-4fb2-a7d2-1cd6cfaa5557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:07.008159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5301b4d-3339-4fb2-a7d2-1cd6cfaa5557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:07.008916 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5301b4d-3339-4fb2-a7d2-1cd6cfaa5557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:07.010583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5301b4d-3339-4fb2-a7d2-1cd6cfaa5557 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:07.027624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1217/2430] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:07.031732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf69fe3-d794-46b1-ab18-0b2238a4e87e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:07.042413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf69fe3-d794-46b1-ab18-0b2238a4e87e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:07.047386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37d3e3-e15c-4392-8e40-69034262393f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:07.048263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37d3e3-e15c-4392-8e40-69034262393f admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:07.048560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37d3e3-e15c-4392-8e40-69034262393f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:07.048817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37d3e3-e15c-4392-8e40-69034262393f admin admin] PATH_INFO: `/v3/projects/f18054f102d64f648894c00fb910e982/users/7b5b1c87da9a439badb6abe9a388a5a8/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:07.063494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37d3e3-e15c-4392-8e40-69034262393f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18054f102d64f648894c00fb910e982, user_id=7b5b1c87da9a439badb6abe9a388a5a8, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:07.066173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37d3e3-e15c-4392-8e40-69034262393f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:07.080301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1214/2431] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:07 2022] PUT /identity/v3/projects/f18054f102d64f648894c00fb910e982/users/7b5b1c87da9a439badb6abe9a388a5a8/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:14:07.084407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5301b4d-3339-4fb2-a7d2-1cd6cfaa5557 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:07.095161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5301b4d-3339-4fb2-a7d2-1cd6cfaa5557 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:07.124225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:07.154918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b8a5f-c1b1-4288-a3a2-bf4736f00be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:07.155583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b8a5f-c1b1-4288-a3a2-bf4736f00be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:07.155872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b8a5f-c1b1-4288-a3a2-bf4736f00be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:07.156154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b8a5f-c1b1-4288-a3a2-bf4736f00be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:07.156875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b8a5f-c1b1-4288-a3a2-bf4736f00be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:07.158546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b8a5f-c1b1-4288-a3a2-bf4736f00be1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:07.168155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1218/2432] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:07.172446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37d3e3-e15c-4392-8e40-69034262393f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:07.183129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37d3e3-e15c-4392-8e40-69034262393f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:07.187706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46718f4b-a3ca-4313-858b-230741d8f30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:07.188579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46718f4b-a3ca-4313-858b-230741d8f30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:07.188923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46718f4b-a3ca-4313-858b-230741d8f30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:07.189261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46718f4b-a3ca-4313-858b-230741d8f30d admin admin] PATH_INFO: `/v3/projects/f18054f102d64f648894c00fb910e982/users/7b5b1c87da9a439badb6abe9a388a5a8/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:07.191269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46718f4b-a3ca-4313-858b-230741d8f30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18054f102d64f648894c00fb910e982, user_id=7b5b1c87da9a439badb6abe9a388a5a8, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:07.194213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46718f4b-a3ca-4313-858b-230741d8f30d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:07.205339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1215/2433] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:07 2022] PUT /identity/v3/projects/f18054f102d64f648894c00fb910e982/users/7b5b1c87da9a439badb6abe9a388a5a8/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:07.217407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73231955-9f34-4751-aa80-4f3c901cec1f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:07.217701 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73231955-9f34-4751-aa80-4f3c901cec1f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:07.217984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73231955-9f34-4751-aa80-4f3c901cec1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:07.244227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-73231955-9f34-4751-aa80-4f3c901cec1f None None] MFA Rules not processed for user `7b5b1c87da9a439badb6abe9a388a5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:07.245514 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-73231955-9f34-4751-aa80-4f3c901cec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:07.312408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1219/2434] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:07 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:07.324613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46718f4b-a3ca-4313-858b-230741d8f30d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:07.326513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-46718f4b-a3ca-4313-858b-230741d8f30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:07.357694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:07.384885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46718f4b-a3ca-4313-858b-230741d8f30d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:07.388535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c286d30-6543-4511-a3e4-245c8cfb976a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:07.389208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c286d30-6543-4511-a3e4-245c8cfb976a service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:07.389504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c286d30-6543-4511-a3e4-245c8cfb976a service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:07.389788 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c286d30-6543-4511-a3e4-245c8cfb976a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:07.399320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c286d30-6543-4511-a3e4-245c8cfb976a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:07.400758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c286d30-6543-4511-a3e4-245c8cfb976a service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:07.410325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1216/2435] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:07 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:07.485852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73231955-9f34-4751-aa80-4f3c901cec1f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:07.496149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73231955-9f34-4751-aa80-4f3c901cec1f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:07.499831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dc0aa-a1a5-41dc-83fd-49bcea27d51a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:07.500468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dc0aa-a1a5-41dc-83fd-49bcea27d51a service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:07.500761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dc0aa-a1a5-41dc-83fd-49bcea27d51a service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:07.501045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dc0aa-a1a5-41dc-83fd-49bcea27d51a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:07.558007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dc0aa-a1a5-41dc-83fd-49bcea27d51a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:07.559452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dc0aa-a1a5-41dc-83fd-49bcea27d51a service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:07.568568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1220/2436] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:07 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:17.406039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=14 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:17.564383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=77 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:22.268001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c286d30-6543-4511-a3e4-245c8cfb976a service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:22.344050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c286d30-6543-4511-a3e4-245c8cfb976a service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:22.348974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:22.349933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:22.350221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:22.350504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] PATH_INFO: `/v3/users/cbef6b12e954408abe4ee1f7ea3e898f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:22.351708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbef6b12e954408abe4ee1f7ea3e898f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:22.354305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:22.401465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbef6b12e954408abe4ee1f7ea3e898f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:22.413799 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbef6b12e954408abe4ee1f7ea3e898f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:22.423510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbef6b12e954408abe4ee1f7ea3e898f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:22.431527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbef6b12e954408abe4ee1f7ea3e898f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:22.432210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbef6b12e954408abe4ee1f7ea3e898f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:22.438216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1217/2437] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:22 2022] DELETE /identity/v3/users/cbef6b12e954408abe4ee1f7ea3e898f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:14:22.442843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0689ad-8c87-400f-96d7-72688463bccd None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:22.443326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0689ad-8c87-400f-96d7-72688463bccd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:22.443424 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0689ad-8c87-400f-96d7-72688463bccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:22.460269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8e0689ad-8c87-400f-96d7-72688463bccd None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:22.461541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8e0689ad-8c87-400f-96d7-72688463bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:22.523673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1221/2438] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:22.534396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:22.535832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:22.598265 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf537d4d-f7bb-4520-aaa4-bdfa4b90848d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:22.602527 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebd7d1-f624-4213-90a1-bc287f81ac09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:22.603902 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd7d1-f624-4213-90a1-bc287f81ac09 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:22.604189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd7d1-f624-4213-90a1-bc287f81ac09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:22.604526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd7d1-f624-4213-90a1-bc287f81ac09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:22.614581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebd7d1-f624-4213-90a1-bc287f81ac09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:22.616026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebd7d1-f624-4213-90a1-bc287f81ac09 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:22.626124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1218/2439] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:22.907979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0689ad-8c87-400f-96d7-72688463bccd None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:22.909980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8e0689ad-8c87-400f-96d7-72688463bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:22.933515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0689ad-8c87-400f-96d7-72688463bccd None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:22.939726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:22.940656 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:22.940967 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:22.941316 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] PATH_INFO: `/v3/projects/9bc58686e6914cf9b5cc10604ca04d39` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:22.942850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc58686e6914cf9b5cc10604ca04d39)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:22.944769 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:22.993987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc58686e6914cf9b5cc10604ca04d39', 'request_id': 'req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.000420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc58686e6914cf9b5cc10604ca04d39', 'request_id': 'req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.007928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc58686e6914cf9b5cc10604ca04d39', 'request_id': 'req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.020001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] The token cache is being invalidate because project 9bc58686e6914cf9b5cc10604ca04d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:23.020317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.020968 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.022689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1222/2440] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:22 2022] DELETE /identity/v3/projects/9bc58686e6914cf9b5cc10604ca04d39 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:23.027289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebd7d1-f624-4213-90a1-bc287f81ac09 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:23.037274 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebd7d1-f624-4213-90a1-bc287f81ac09 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:23.055492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:23.100900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:23.101802 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:23.102088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:23.102418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] PATH_INFO: `/v3/users/7b5b1c87da9a439badb6abe9a388a5a8` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:23.103672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5b1c87da9a439badb6abe9a388a5a8)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:23.105757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:23.156356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5b1c87da9a439badb6abe9a388a5a8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.170116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5b1c87da9a439badb6abe9a388a5a8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.177744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5b1c87da9a439badb6abe9a388a5a8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.185760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5b1c87da9a439badb6abe9a388a5a8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.186402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c4a3bf95-5509-47fc-bb85-ef59fa2ca465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5b1c87da9a439badb6abe9a388a5a8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.192322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1219/2441] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:23 2022] DELETE /identity/v3/users/7b5b1c87da9a439badb6abe9a388a5a8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:23.483929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:23.485478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:23.513229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8fd83-fc5f-44c9-a7b6-a0dd4c55db14 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:23.574189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:23.574965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:23.575249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:23.575525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] PATH_INFO: `/v3/projects/f18054f102d64f648894c00fb910e982` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:23.576731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18054f102d64f648894c00fb910e982)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:23.578673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:23.610908 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:23.629688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18054f102d64f648894c00fb910e982', 'request_id': 'req-a45566ca-48f0-426b-a8fd-65a0083f79ba'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.635954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18054f102d64f648894c00fb910e982', 'request_id': 'req-a45566ca-48f0-426b-a8fd-65a0083f79ba'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.643003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18054f102d64f648894c00fb910e982', 'request_id': 'req-a45566ca-48f0-426b-a8fd-65a0083f79ba'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.653344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] The token cache is being invalidate because project f18054f102d64f648894c00fb910e982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:23.653627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.654357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a45566ca-48f0-426b-a8fd-65a0083f79ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:23.655876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1223/2442] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:23 2022] DELETE /identity/v3/projects/f18054f102d64f648894c00fb910e982 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:23.660687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f2795-36da-4400-86dc-cfba5f4ebfd9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:23.661227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f2795-36da-4400-86dc-cfba5f4ebfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:23.661344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f2795-36da-4400-86dc-cfba5f4ebfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:23.679732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fb3f2795-36da-4400-86dc-cfba5f4ebfd9 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:23.681553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fb3f2795-36da-4400-86dc-cfba5f4ebfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:23.741747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1220/2443] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:23.750101 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2aae7a-f12c-46a7-95fa-aacf873555a9 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:23.750390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2aae7a-f12c-46a7-95fa-aacf873555a9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:23.750672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2aae7a-f12c-46a7-95fa-aacf873555a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:23.768928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0e2aae7a-f12c-46a7-95fa-aacf873555a9 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:23.770162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0e2aae7a-f12c-46a7-95fa-aacf873555a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:23.778174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1224/2444] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:23.782295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f2795-36da-4400-86dc-cfba5f4ebfd9 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:23.792434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f2795-36da-4400-86dc-cfba5f4ebfd9 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:23.797051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a03949-eb82-4a5a-9699-e2a358a653fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:23.797733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a03949-eb82-4a5a-9699-e2a358a653fa admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:23.798012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a03949-eb82-4a5a-9699-e2a358a653fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:23.798289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a03949-eb82-4a5a-9699-e2a358a653fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:23.799006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a03949-eb82-4a5a-9699-e2a358a653fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:14:23.799311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a03949-eb82-4a5a-9699-e2a358a653fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:23.800901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a03949-eb82-4a5a-9699-e2a358a653fa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:23.812902 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1221/2445] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:14:23.817239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2aae7a-f12c-46a7-95fa-aacf873555a9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:23.826780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2aae7a-f12c-46a7-95fa-aacf873555a9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:23.831390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa92a2-3dc4-4ef2-b5d5-c7d5c9784846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:23.832029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa92a2-3dc4-4ef2-b5d5-c7d5c9784846 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:23.832307 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa92a2-3dc4-4ef2-b5d5-c7d5c9784846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:23.832580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa92a2-3dc4-4ef2-b5d5-c7d5c9784846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:23.833298 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa92a2-3dc4-4ef2-b5d5-c7d5c9784846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:23.835173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa92a2-3dc4-4ef2-b5d5-c7d5c9784846 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:23.860175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1225/2446] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:23 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:23.864250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a03949-eb82-4a5a-9699-e2a358a653fa admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:23.874736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a03949-eb82-4a5a-9699-e2a358a653fa admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:23.899635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:23.931378 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a33e33-d43b-4cbe-95cc-e6cb01a1f7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:23.932049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a33e33-d43b-4cbe-95cc-e6cb01a1f7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:23.932333 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a33e33-d43b-4cbe-95cc-e6cb01a1f7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:23.932610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a33e33-d43b-4cbe-95cc-e6cb01a1f7d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:23.933333 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a33e33-d43b-4cbe-95cc-e6cb01a1f7d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:23.935259 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a33e33-d43b-4cbe-95cc-e6cb01a1f7d9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:23.958670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1222/2447] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:23 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:23.959509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa92a2-3dc4-4ef2-b5d5-c7d5c9784846 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:23.970480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa92a2-3dc4-4ef2-b5d5-c7d5c9784846 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:23.975100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee348759-29e2-4989-a6fe-4be625ff6174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:23.975762 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee348759-29e2-4989-a6fe-4be625ff6174 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:23.976053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee348759-29e2-4989-a6fe-4be625ff6174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:23.976335 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee348759-29e2-4989-a6fe-4be625ff6174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:23.977060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee348759-29e2-4989-a6fe-4be625ff6174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:23.979266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee348759-29e2-4989-a6fe-4be625ff6174 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:23.989879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1226/2448] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:23.994085 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a33e33-d43b-4cbe-95cc-e6cb01a1f7d9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:24.004405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a33e33-d43b-4cbe-95cc-e6cb01a1f7d9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:24.009244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe2f87-619c-4b8e-90e2-6a83fa181b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:24.009902 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe2f87-619c-4b8e-90e2-6a83fa181b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.010193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe2f87-619c-4b8e-90e2-6a83fa181b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.010474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe2f87-619c-4b8e-90e2-6a83fa181b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.011203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe2f87-619c-4b8e-90e2-6a83fa181b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:24.012801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe2f87-619c-4b8e-90e2-6a83fa181b11 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:24.023294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1223/2449] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:24.027983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee348759-29e2-4989-a6fe-4be625ff6174 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:24.038360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee348759-29e2-4989-a6fe-4be625ff6174 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:24.043682 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe17fc-3ce0-4e7f-a45e-4adc53572558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:24.044558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe17fc-3ce0-4e7f-a45e-4adc53572558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.044845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe17fc-3ce0-4e7f-a45e-4adc53572558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.045153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe17fc-3ce0-4e7f-a45e-4adc53572558 admin admin] PATH_INFO: `/v3/projects/ce954fa5c18b4b2a81cf4120434f8173/users/2bce3864cf014feba09e98b25b9444a5/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.064921 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe17fc-3ce0-4e7f-a45e-4adc53572558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce954fa5c18b4b2a81cf4120434f8173, user_id=2bce3864cf014feba09e98b25b9444a5, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:24.067921 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe17fc-3ce0-4e7f-a45e-4adc53572558 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:24.081826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1227/2450] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:24 2022] PUT /identity/v3/projects/ce954fa5c18b4b2a81cf4120434f8173/users/2bce3864cf014feba09e98b25b9444a5/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:24.093508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1527d-08e5-4770-9be2-30c49b8f7374 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.093807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1527d-08e5-4770-9be2-30c49b8f7374 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.094091 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1527d-08e5-4770-9be2-30c49b8f7374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.122907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-62b1527d-08e5-4770-9be2-30c49b8f7374 None None] MFA Rules not processed for user `2bce3864cf014feba09e98b25b9444a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:24.124151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-62b1527d-08e5-4770-9be2-30c49b8f7374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:24.195383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1224/2451] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:24.199965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe17fc-3ce0-4e7f-a45e-4adc53572558 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:24.269712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe17fc-3ce0-4e7f-a45e-4adc53572558 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:24.275276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30776953-88cf-457a-9c9f-b2675fceea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:24.276076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30776953-88cf-457a-9c9f-b2675fceea69 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.276363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30776953-88cf-457a-9c9f-b2675fceea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.276640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30776953-88cf-457a-9c9f-b2675fceea69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.277389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30776953-88cf-457a-9c9f-b2675fceea69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:24.279334 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30776953-88cf-457a-9c9f-b2675fceea69 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:24.306761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1228/2452] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:24 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:24.311084 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1527d-08e5-4770-9be2-30c49b8f7374 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:24.326595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1527d-08e5-4770-9be2-30c49b8f7374 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:24.388641 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759447-053e-4c1b-ab31-3f6531823b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:24.389516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759447-053e-4c1b-ab31-3f6531823b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.389858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759447-053e-4c1b-ab31-3f6531823b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.390137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759447-053e-4c1b-ab31-3f6531823b9c admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.391032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6759447-053e-4c1b-ab31-3f6531823b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:24.392976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6759447-053e-4c1b-ab31-3f6531823b9c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:24.414145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1225/2453] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:24 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:24.418716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30776953-88cf-457a-9c9f-b2675fceea69 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:24.432056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30776953-88cf-457a-9c9f-b2675fceea69 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:24.436877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52b9de-0746-4f6d-b39d-435f124516d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:24.442812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52b9de-0746-4f6d-b39d-435f124516d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.444695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52b9de-0746-4f6d-b39d-435f124516d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.444983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52b9de-0746-4f6d-b39d-435f124516d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.445770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52b9de-0746-4f6d-b39d-435f124516d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:24.447897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52b9de-0746-4f6d-b39d-435f124516d6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:24.460871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1229/2454] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:14:24.465930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759447-053e-4c1b-ab31-3f6531823b9c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:24.478807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759447-053e-4c1b-ab31-3f6531823b9c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:24.494442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf3b1a-bb7e-4869-94c8-b5b78cc96932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:24.495489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf3b1a-bb7e-4869-94c8-b5b78cc96932 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.495876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf3b1a-bb7e-4869-94c8-b5b78cc96932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.496301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf3b1a-bb7e-4869-94c8-b5b78cc96932 admin admin] PATH_INFO: `/v3/projects/e079c5ff6aeb465a88f8158fd354a882/users/9cd7cad98d384cb6a84a73715e740adf/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.521727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf3b1a-bb7e-4869-94c8-b5b78cc96932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e079c5ff6aeb465a88f8158fd354a882, user_id=9cd7cad98d384cb6a84a73715e740adf, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:24.525666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf3b1a-bb7e-4869-94c8-b5b78cc96932 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:24.544138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1226/2455] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:24 2022] PUT /identity/v3/projects/e079c5ff6aeb465a88f8158fd354a882/users/9cd7cad98d384cb6a84a73715e740adf/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:24.548076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52b9de-0746-4f6d-b39d-435f124516d6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:24.570458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52b9de-0746-4f6d-b39d-435f124516d6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:24.610375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:24.652703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03634d-0be3-469d-bbb4-d0cdd0e2a781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:24.653412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03634d-0be3-469d-bbb4-d0cdd0e2a781 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.653748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03634d-0be3-469d-bbb4-d0cdd0e2a781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.653980 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03634d-0be3-469d-bbb4-d0cdd0e2a781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.654712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03634d-0be3-469d-bbb4-d0cdd0e2a781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:24.656331 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03634d-0be3-469d-bbb4-d0cdd0e2a781 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:24.669381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1230/2456] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:24.673473 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf3b1a-bb7e-4869-94c8-b5b78cc96932 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:24.680440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:24.687972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf3b1a-bb7e-4869-94c8-b5b78cc96932 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:24.693376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ef6a2-335c-4c66-9652-5438b9c7f392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:24.694324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef6a2-335c-4c66-9652-5438b9c7f392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.694608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef6a2-335c-4c66-9652-5438b9c7f392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.694886 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef6a2-335c-4c66-9652-5438b9c7f392 admin admin] PATH_INFO: `/v3/projects/e079c5ff6aeb465a88f8158fd354a882/users/9cd7cad98d384cb6a84a73715e740adf/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.697507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ef6a2-335c-4c66-9652-5438b9c7f392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e079c5ff6aeb465a88f8158fd354a882, user_id=9cd7cad98d384cb6a84a73715e740adf, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:24.700154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ef6a2-335c-4c66-9652-5438b9c7f392 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:24.711134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1227/2457] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:24 2022] PUT /identity/v3/projects/e079c5ff6aeb465a88f8158fd354a882/users/9cd7cad98d384cb6a84a73715e740adf/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:24.721466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7f17b-db69-42f6-aff1-49de9efef305 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.721759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7f17b-db69-42f6-aff1-49de9efef305 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.722039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7f17b-db69-42f6-aff1-49de9efef305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.752867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4bb7f17b-db69-42f6-aff1-49de9efef305 None None] MFA Rules not processed for user `9cd7cad98d384cb6a84a73715e740adf`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:24.754158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4bb7f17b-db69-42f6-aff1-49de9efef305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:24.834399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1231/2458] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:24.846724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ef6a2-335c-4c66-9652-5438b9c7f392 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:24.848689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4e2ef6a2-335c-4c66-9652-5438b9c7f392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:24.916145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ef6a2-335c-4c66-9652-5438b9c7f392 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:24.919893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c61e0e-4e80-4400-aa21-8a36ef27f638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:24.920522 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c61e0e-4e80-4400-aa21-8a36ef27f638 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:24.920851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c61e0e-4e80-4400-aa21-8a36ef27f638 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:24.921341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c61e0e-4e80-4400-aa21-8a36ef27f638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:24.933175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c61e0e-4e80-4400-aa21-8a36ef27f638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:24.934680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c61e0e-4e80-4400-aa21-8a36ef27f638 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:24.953472 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1228/2459] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:26.321209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7f17b-db69-42f6-aff1-49de9efef305 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:26.387391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7f17b-db69-42f6-aff1-49de9efef305 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:26.391892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:26.392760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:26.393041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:26.393348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] PATH_INFO: `/v3/users/2bce3864cf014feba09e98b25b9444a5` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:26.394533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bce3864cf014feba09e98b25b9444a5)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:26.396430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:26.444203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bce3864cf014feba09e98b25b9444a5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:26.445351 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:26.457484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bce3864cf014feba09e98b25b9444a5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:26.466061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bce3864cf014feba09e98b25b9444a5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:26.466673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bce3864cf014feba09e98b25b9444a5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:26.473986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bce3864cf014feba09e98b25b9444a5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:26.480748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1232/2460] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:26 2022] DELETE /identity/v3/users/2bce3864cf014feba09e98b25b9444a5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:26.485824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47932c5c-5ace-4df5-8850-de4b80cd8a32 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:26.486310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47932c5c-5ace-4df5-8850-de4b80cd8a32 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:26.486409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47932c5c-5ace-4df5-8850-de4b80cd8a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:26.505798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-47932c5c-5ace-4df5-8850-de4b80cd8a32 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:26.507040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-47932c5c-5ace-4df5-8850-de4b80cd8a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:26.568214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1229/2461] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:26.578595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:26.580112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:26.639804 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7318d04-ec57-4456-8e3a-5c5ee8c21611 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:26.643326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed62919-4938-49b6-9e20-16b3c234b77b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:26.643972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed62919-4938-49b6-9e20-16b3c234b77b service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:26.644255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed62919-4938-49b6-9e20-16b3c234b77b service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:26.644529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed62919-4938-49b6-9e20-16b3c234b77b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:26.654358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed62919-4938-49b6-9e20-16b3c234b77b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:26.655833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed62919-4938-49b6-9e20-16b3c234b77b service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:26.665821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1233/2462] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:26.950652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47932c5c-5ace-4df5-8850-de4b80cd8a32 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:26.952120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-47932c5c-5ace-4df5-8850-de4b80cd8a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:26.967046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47932c5c-5ace-4df5-8850-de4b80cd8a32 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:26.971923 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:26.972845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:26.973327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:26.973516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] PATH_INFO: `/v3/projects/ce954fa5c18b4b2a81cf4120434f8173` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:26.975572 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce954fa5c18b4b2a81cf4120434f8173)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:26.977667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:27.028507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce954fa5c18b4b2a81cf4120434f8173', 'request_id': 'req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.035182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce954fa5c18b4b2a81cf4120434f8173', 'request_id': 'req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.043624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce954fa5c18b4b2a81cf4120434f8173', 'request_id': 'req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.054440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] The token cache is being invalidate because project ce954fa5c18b4b2a81cf4120434f8173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:27.054755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.055455 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.057605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1230/2463] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:26 2022] DELETE /identity/v3/projects/ce954fa5c18b4b2a81cf4120434f8173 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:27.061689 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed62919-4938-49b6-9e20-16b3c234b77b service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:27.072711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed62919-4938-49b6-9e20-16b3c234b77b service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:27.135825 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:27.137081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:27.137401 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:27.137683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] PATH_INFO: `/v3/users/9cd7cad98d384cb6a84a73715e740adf` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:27.138902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd7cad98d384cb6a84a73715e740adf)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:27.140832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:27.186946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd7cad98d384cb6a84a73715e740adf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.204458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd7cad98d384cb6a84a73715e740adf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.208214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:27.213027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd7cad98d384cb6a84a73715e740adf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.214346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd7cad98d384cb6a84a73715e740adf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.223570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-23dedff1-4659-480a-b995-3fdb0fdf548c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd7cad98d384cb6a84a73715e740adf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.231558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1234/2464] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:27 2022] DELETE /identity/v3/users/9cd7cad98d384cb6a84a73715e740adf => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:27.517530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:27.519411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:27.536336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db4dc3-8699-47aa-8b7f-4e5686ad5f86 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:27.551011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:27.605331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:27.606352 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:27.606805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:27.607275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] PATH_INFO: `/v3/projects/e079c5ff6aeb465a88f8158fd354a882` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:27.609325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e079c5ff6aeb465a88f8158fd354a882)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:27.612466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:27.662931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e079c5ff6aeb465a88f8158fd354a882', 'request_id': 'req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.670145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e079c5ff6aeb465a88f8158fd354a882', 'request_id': 'req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.684471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e079c5ff6aeb465a88f8158fd354a882', 'request_id': 'req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.695737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] The token cache is being invalidate because project e079c5ff6aeb465a88f8158fd354a882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:27.696035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.696723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4cdaf78-cd17-4a31-adea-6cfa2a6f22f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:27.698357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1231/2465] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:27 2022] DELETE /identity/v3/projects/e079c5ff6aeb465a88f8158fd354a882 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:27.703517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcf525-3c83-4607-8a59-67c32407438c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:27.704041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcf525-3c83-4607-8a59-67c32407438c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:27.704157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcf525-3c83-4607-8a59-67c32407438c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:27.724034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-18dcf525-3c83-4607-8a59-67c32407438c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:27.725229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-18dcf525-3c83-4607-8a59-67c32407438c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:27.791555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1235/2466] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:27.799621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4f62-2a49-4336-9615-935c779dcffd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:27.799918 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4f62-2a49-4336-9615-935c779dcffd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:27.800201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4f62-2a49-4336-9615-935c779dcffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:27.819147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fc8c4f62-2a49-4336-9615-935c779dcffd None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:27.820389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fc8c4f62-2a49-4336-9615-935c779dcffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:27.828620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1232/2467] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:27.833281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcf525-3c83-4607-8a59-67c32407438c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:27.843840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcf525-3c83-4607-8a59-67c32407438c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:27.848509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff710061-31f8-4c3a-a5a5-18fa15d9dbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:27.849199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff710061-31f8-4c3a-a5a5-18fa15d9dbb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:27.849495 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff710061-31f8-4c3a-a5a5-18fa15d9dbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:27.849779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff710061-31f8-4c3a-a5a5-18fa15d9dbb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:27.850536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff710061-31f8-4c3a-a5a5-18fa15d9dbb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:14:27.850850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff710061-31f8-4c3a-a5a5-18fa15d9dbb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:27.852457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff710061-31f8-4c3a-a5a5-18fa15d9dbb6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:27.865226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1236/2468] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:14:27.869982 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4f62-2a49-4336-9615-935c779dcffd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:27.879452 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4f62-2a49-4336-9615-935c779dcffd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:27.884610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d8fbe-42a3-4993-af3a-3b4e19e4c630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:27.885338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d8fbe-42a3-4993-af3a-3b4e19e4c630 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:27.885768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d8fbe-42a3-4993-af3a-3b4e19e4c630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:27.886059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d8fbe-42a3-4993-af3a-3b4e19e4c630 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:27.886781 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d8fbe-42a3-4993-af3a-3b4e19e4c630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:27.888717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d8fbe-42a3-4993-af3a-3b4e19e4c630 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:27.922137 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1233/2469] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:27 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:14:27.923356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff710061-31f8-4c3a-a5a5-18fa15d9dbb6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:27.935092 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff710061-31f8-4c3a-a5a5-18fa15d9dbb6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:27.999584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ef8f8-1610-4177-b9ec-4861b57adbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.000176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ef8f8-1610-4177-b9ec-4861b57adbed admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.000462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ef8f8-1610-4177-b9ec-4861b57adbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.000743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ef8f8-1610-4177-b9ec-4861b57adbed admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.001508 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ef8f8-1610-4177-b9ec-4861b57adbed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.003554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ef8f8-1610-4177-b9ec-4861b57adbed admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.025999 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1237/2470] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:27 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:28.030282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d8fbe-42a3-4993-af3a-3b4e19e4c630 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.042166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d8fbe-42a3-4993-af3a-3b4e19e4c630 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:28.047415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c17e-4b14-454c-9bcf-7b7071c2fd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.048161 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c17e-4b14-454c-9bcf-7b7071c2fd63 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.048483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c17e-4b14-454c-9bcf-7b7071c2fd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.048752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c17e-4b14-454c-9bcf-7b7071c2fd63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.049563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279c17e-4b14-454c-9bcf-7b7071c2fd63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.051179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279c17e-4b14-454c-9bcf-7b7071c2fd63 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.062917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1234/2471] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:28.067232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ef8f8-1610-4177-b9ec-4861b57adbed admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.078767 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ef8f8-1610-4177-b9ec-4861b57adbed admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:28.083886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0f95a-3a36-4034-a2b6-36a60a280787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.084618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0f95a-3a36-4034-a2b6-36a60a280787 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.084897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0f95a-3a36-4034-a2b6-36a60a280787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.085236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0f95a-3a36-4034-a2b6-36a60a280787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.086135 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0f95a-3a36-4034-a2b6-36a60a280787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.088411 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0f95a-3a36-4034-a2b6-36a60a280787 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.102653 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1238/2472] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:28.104316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c17e-4b14-454c-9bcf-7b7071c2fd63 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.115554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c17e-4b14-454c-9bcf-7b7071c2fd63 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:28.120564 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13359a68-ee72-4bcb-9a4f-1f6bd1344002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.121494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13359a68-ee72-4bcb-9a4f-1f6bd1344002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.121945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13359a68-ee72-4bcb-9a4f-1f6bd1344002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.122072 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13359a68-ee72-4bcb-9a4f-1f6bd1344002 admin admin] PATH_INFO: `/v3/projects/14d3650acc0c462eb0aaca8fba7e19aa/users/fc4c1453eb294b7590e56e7ce3af7800/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.140878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13359a68-ee72-4bcb-9a4f-1f6bd1344002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d3650acc0c462eb0aaca8fba7e19aa, user_id=fc4c1453eb294b7590e56e7ce3af7800, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.143914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13359a68-ee72-4bcb-9a4f-1f6bd1344002 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.159792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1235/2473] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:28 2022] PUT /identity/v3/projects/14d3650acc0c462eb0aaca8fba7e19aa/users/fc4c1453eb294b7590e56e7ce3af7800/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:28.171360 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56249bbb-be7e-4723-a6d1-d7add3dadadb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.171722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56249bbb-be7e-4723-a6d1-d7add3dadadb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.172081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56249bbb-be7e-4723-a6d1-d7add3dadadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.200132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-56249bbb-be7e-4723-a6d1-d7add3dadadb None None] MFA Rules not processed for user `fc4c1453eb294b7590e56e7ce3af7800`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:28.201493 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-56249bbb-be7e-4723-a6d1-d7add3dadadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:28.245522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:28.272078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1239/2474] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:28.277344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13359a68-ee72-4bcb-9a4f-1f6bd1344002 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.333266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:28.355949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13359a68-ee72-4bcb-9a4f-1f6bd1344002 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:28.361294 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d6a96-06f6-4439-927d-3c66e358aafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.361957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d6a96-06f6-4439-927d-3c66e358aafd admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.362242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d6a96-06f6-4439-927d-3c66e358aafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.362520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d6a96-06f6-4439-927d-3c66e358aafd admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.363219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d6a96-06f6-4439-927d-3c66e358aafd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.365158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d6a96-06f6-4439-927d-3c66e358aafd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.393448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1236/2475] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:28 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:28.397503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56249bbb-be7e-4723-a6d1-d7add3dadadb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.409616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56249bbb-be7e-4723-a6d1-d7add3dadadb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:28.471846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f70fec-9459-4b73-a223-38c5d062143c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.472533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f70fec-9459-4b73-a223-38c5d062143c admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.472935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f70fec-9459-4b73-a223-38c5d062143c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.473133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f70fec-9459-4b73-a223-38c5d062143c admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.473846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f70fec-9459-4b73-a223-38c5d062143c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.475815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f70fec-9459-4b73-a223-38c5d062143c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.497403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1240/2476] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:28 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:14:28.501577 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d6a96-06f6-4439-927d-3c66e358aafd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.512965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d6a96-06f6-4439-927d-3c66e358aafd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:28.518592 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767d65d-de11-40ac-bdaa-98ff09c87317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.519411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767d65d-de11-40ac-bdaa-98ff09c87317 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.519563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767d65d-de11-40ac-bdaa-98ff09c87317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.519829 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767d65d-de11-40ac-bdaa-98ff09c87317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.520686 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767d65d-de11-40ac-bdaa-98ff09c87317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.522251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767d65d-de11-40ac-bdaa-98ff09c87317 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.533891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1237/2477] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:28.538110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f70fec-9459-4b73-a223-38c5d062143c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.548458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f70fec-9459-4b73-a223-38c5d062143c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:28.553546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062c782-76c6-4db4-b7bf-a158bc04e8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.554454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062c782-76c6-4db4-b7bf-a158bc04e8a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.554739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062c782-76c6-4db4-b7bf-a158bc04e8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.555017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062c782-76c6-4db4-b7bf-a158bc04e8a7 admin admin] PATH_INFO: `/v3/projects/7b125b09df1f4c12bc773103b95f6e08/users/83f4edfb164248fa9515a894854f3602/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.572060 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062c782-76c6-4db4-b7bf-a158bc04e8a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b125b09df1f4c12bc773103b95f6e08, user_id=83f4edfb164248fa9515a894854f3602, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.574829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062c782-76c6-4db4-b7bf-a158bc04e8a7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.588340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1241/2478] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:28 2022] PUT /identity/v3/projects/7b125b09df1f4c12bc773103b95f6e08/users/83f4edfb164248fa9515a894854f3602/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:28.592284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767d65d-de11-40ac-bdaa-98ff09c87317 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.602779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767d65d-de11-40ac-bdaa-98ff09c87317 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:28.668140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e667c6-c205-4c03-8030-eea5ddd84da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.668818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e667c6-c205-4c03-8030-eea5ddd84da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.669129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e667c6-c205-4c03-8030-eea5ddd84da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.669420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e667c6-c205-4c03-8030-eea5ddd84da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.670159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e667c6-c205-4c03-8030-eea5ddd84da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.671768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e667c6-c205-4c03-8030-eea5ddd84da2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.683609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1238/2479] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:28.686989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062c782-76c6-4db4-b7bf-a158bc04e8a7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.698955 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062c782-76c6-4db4-b7bf-a158bc04e8a7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:28.703734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cdc75-e20a-4d3c-b5e6-c31fc2de24b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:28.704783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cdc75-e20a-4d3c-b5e6-c31fc2de24b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.704978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cdc75-e20a-4d3c-b5e6-c31fc2de24b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.705279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cdc75-e20a-4d3c-b5e6-c31fc2de24b0 admin admin] PATH_INFO: `/v3/projects/7b125b09df1f4c12bc773103b95f6e08/users/83f4edfb164248fa9515a894854f3602/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.707546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cdc75-e20a-4d3c-b5e6-c31fc2de24b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b125b09df1f4c12bc773103b95f6e08, user_id=83f4edfb164248fa9515a894854f3602, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:28.710117 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cdc75-e20a-4d3c-b5e6-c31fc2de24b0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:28.722954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1242/2480] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:28 2022] PUT /identity/v3/projects/7b125b09df1f4c12bc773103b95f6e08/users/83f4edfb164248fa9515a894854f3602/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:14:28.733667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0042-1074-4a9b-970c-e99577648369 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:28.734026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0042-1074-4a9b-970c-e99577648369 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:28.734316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0042-1074-4a9b-970c-e99577648369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:28.766211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a83a0042-1074-4a9b-970c-e99577648369 None None] MFA Rules not processed for user `83f4edfb164248fa9515a894854f3602`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:28.768253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a83a0042-1074-4a9b-970c-e99577648369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:28.884861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1239/2481] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:14:28.896969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cdc75-e20a-4d3c-b5e6-c31fc2de24b0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:28.898773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-443cdc75-e20a-4d3c-b5e6-c31fc2de24b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:29.003927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cdc75-e20a-4d3c-b5e6-c31fc2de24b0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:29.008656 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ccd2a-40d4-41c8-a8a4-db4dd07624ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:29.009710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ccd2a-40d4-41c8-a8a4-db4dd07624ef service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:29.010033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ccd2a-40d4-41c8-a8a4-db4dd07624ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:29.010415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ccd2a-40d4-41c8-a8a4-db4dd07624ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:29.071486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:29.086310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ccd2a-40d4-41c8-a8a4-db4dd07624ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:29.087873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ccd2a-40d4-41c8-a8a4-db4dd07624ef service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:29.098926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1243/2482] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:35.860689 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a0042-1074-4a9b-970c-e99577648369 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:35.862139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a83a0042-1074-4a9b-970c-e99577648369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:35.874750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a0042-1074-4a9b-970c-e99577648369 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:35.878335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f4a8a-822b-4724-b022-8e34a947c2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:35.878998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f4a8a-822b-4724-b022-8e34a947c2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:35.879437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f4a8a-822b-4724-b022-8e34a947c2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:35.879723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f4a8a-822b-4724-b022-8e34a947c2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:35.889544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f4a8a-822b-4724-b022-8e34a947c2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:35.891050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f4a8a-822b-4724-b022-8e34a947c2a7 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:35.902255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1240/2483] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:39.094859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:43.632808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ccd2a-40d4-41c8-a8a4-db4dd07624ef service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:43.634342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ff9ccd2a-40d4-41c8-a8a4-db4dd07624ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:43.691400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ccd2a-40d4-41c8-a8a4-db4dd07624ef service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:43.695399 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e4bc4-987a-48c5-a97e-3c92466c4ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:43.696038 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e4bc4-987a-48c5-a97e-3c92466c4ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:43.696321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e4bc4-987a-48c5-a97e-3c92466c4ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:43.696596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e4bc4-987a-48c5-a97e-3c92466c4ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:43.698378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f98e4bc4-987a-48c5-a97e-3c92466c4ab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:43.762668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e4bc4-987a-48c5-a97e-3c92466c4ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:43.764116 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e4bc4-987a-48c5-a97e-3c92466c4ab7 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:43.773816 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1244/2484] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:45.896288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:50.043695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f4a8a-822b-4724-b022-8e34a947c2a7 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:50.045716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f02f4a8a-822b-4724-b022-8e34a947c2a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:50.066651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f4a8a-822b-4724-b022-8e34a947c2a7 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:50.071648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:50.072518 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:50.073139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:50.073139 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] PATH_INFO: `/v3/users/569256e6d1d1418eafb5ff2ed8909926` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:50.090065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=569256e6d1d1418eafb5ff2ed8909926)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:50.092385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:50.156490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e4bc4-987a-48c5-a97e-3c92466c4ab7 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:50.161268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '569256e6d1d1418eafb5ff2ed8909926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.177581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '569256e6d1d1418eafb5ff2ed8909926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.181565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e4bc4-987a-48c5-a97e-3c92466c4ab7 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:50.187943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '569256e6d1d1418eafb5ff2ed8909926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.188106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a941c-61bc-4062-9e12-432eb1ca9ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:50.188622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a941c-61bc-4062-9e12-432eb1ca9ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:50.188910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a941c-61bc-4062-9e12-432eb1ca9ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:50.189234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a941c-61bc-4062-9e12-432eb1ca9ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:50.191544 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d37a941c-61bc-4062-9e12-432eb1ca9ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:50.196976 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '569256e6d1d1418eafb5ff2ed8909926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.197763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '569256e6d1d1418eafb5ff2ed8909926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.206075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1241/2485] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:50 2022] DELETE /identity/v3/users/569256e6d1d1418eafb5ff2ed8909926 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:50.206553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a941c-61bc-4062-9e12-432eb1ca9ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:50.208228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a941c-61bc-4062-9e12-432eb1ca9ed4 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:50.222042 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1245/2486] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:50.466234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:50.467842 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:50.570221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d77d3a-7384-4925-bbfd-7acf16a5b053 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:50.577407 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:50.578870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:50.579453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:50.579992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] PATH_INFO: `/v3/projects/cccd9a91f96a4a698f5d9d7028216a35` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:50.592608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cccd9a91f96a4a698f5d9d7028216a35)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:50.594561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:50.645076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cccd9a91f96a4a698f5d9d7028216a35', 'request_id': 'req-13ba3be7-9eca-4482-a295-a9de6ea247f7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.654444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cccd9a91f96a4a698f5d9d7028216a35', 'request_id': 'req-13ba3be7-9eca-4482-a295-a9de6ea247f7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.659052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:50.666391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cccd9a91f96a4a698f5d9d7028216a35', 'request_id': 'req-13ba3be7-9eca-4482-a295-a9de6ea247f7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.681543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] The token cache is being invalidate because project cccd9a91f96a4a698f5d9d7028216a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:50.681838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.682517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.684861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1242/2487] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:50 2022] DELETE /identity/v3/projects/cccd9a91f96a4a698f5d9d7028216a35 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:50.688823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a941c-61bc-4062-9e12-432eb1ca9ed4 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:50.700029 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a941c-61bc-4062-9e12-432eb1ca9ed4 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:50.763441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:50.764461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:50.764742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:50.765023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] PATH_INFO: `/v3/users/d1a8d31ce48f4fb7bd22f4fe98f03565` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:50.780510 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a8d31ce48f4fb7bd22f4fe98f03565)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:50.782546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:50.815034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:50.836268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a8d31ce48f4fb7bd22f4fe98f03565'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.852296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a8d31ce48f4fb7bd22f4fe98f03565'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.860001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a8d31ce48f4fb7bd22f4fe98f03565'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.860894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a8d31ce48f4fb7bd22f4fe98f03565'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.868333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a8d31ce48f4fb7bd22f4fe98f03565'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:50.875639 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1246/2488] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:50 2022] DELETE /identity/v3/users/d1a8d31ce48f4fb7bd22f4fe98f03565 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:51.054727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:51.057136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:51.080483 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba3be7-9eca-4482-a295-a9de6ea247f7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:51.142202 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:51.143120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:51.143412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:51.143694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] PATH_INFO: `/v3/projects/1f9ddf92bc3d46a18803d3493ce69eb9` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:51.154885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f9ddf92bc3d46a18803d3493ce69eb9)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:51.156825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:51.207820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f9ddf92bc3d46a18803d3493ce69eb9', 'request_id': 'req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:51.214713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f9ddf92bc3d46a18803d3493ce69eb9', 'request_id': 'req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:51.223052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f9ddf92bc3d46a18803d3493ce69eb9', 'request_id': 'req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:51.236358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] The token cache is being invalidate because project 1f9ddf92bc3d46a18803d3493ce69eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:51.236729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:51.237866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:51.240031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1243/2489] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:51 2022] DELETE /identity/v3/projects/1f9ddf92bc3d46a18803d3493ce69eb9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:53.675760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:53.677279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:53.691278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7ffb-56ff-4d79-a067-b0dc74aa96bb admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:53.747897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:53.748681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:53.749026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:53.749329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] PATH_INFO: `/v3/users/b6ce650c0ae6494ab895644347bbb5b7` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:53.750504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ce650c0ae6494ab895644347bbb5b7)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:53.752416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:53.800491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ce650c0ae6494ab895644347bbb5b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:53.812203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ce650c0ae6494ab895644347bbb5b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:53.819344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ce650c0ae6494ab895644347bbb5b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:53.819934 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ce650c0ae6494ab895644347bbb5b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:53.827471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f935ce71-f53d-4a57-874d-86f2cc566422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ce650c0ae6494ab895644347bbb5b7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:53.833770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1247/2490] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:53 2022] DELETE /identity/v3/users/b6ce650c0ae6494ab895644347bbb5b7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:54.006645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:54.008127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:54.022733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf96cd-66c8-4a01-b6c9-66e7eccd94fb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:54.082233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:54.083088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.083371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.083647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] PATH_INFO: `/v3/projects/0e052c1c726e48ea98ee7b21c3cd55d7` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.084822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e052c1c726e48ea98ee7b21c3cd55d7)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:54.086756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:54.093682 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:54.134414 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e052c1c726e48ea98ee7b21c3cd55d7', 'request_id': 'req-8f390210-525a-4366-b872-42c2c883038e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:54.140657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e052c1c726e48ea98ee7b21c3cd55d7', 'request_id': 'req-8f390210-525a-4366-b872-42c2c883038e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:54.148368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e052c1c726e48ea98ee7b21c3cd55d7', 'request_id': 'req-8f390210-525a-4366-b872-42c2c883038e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:54.160128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] The token cache is being invalidate because project 0e052c1c726e48ea98ee7b21c3cd55d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:54.160416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:54.161082 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8f390210-525a-4366-b872-42c2c883038e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:54.162850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1244/2491] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:54 2022] DELETE /identity/v3/projects/0e052c1c726e48ea98ee7b21c3cd55d7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:54.168379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcf5fd-0ad6-496c-beeb-e10113782072 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.168669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcf5fd-0ad6-496c-beeb-e10113782072 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.168956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcf5fd-0ad6-496c-beeb-e10113782072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.187457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6adcf5fd-0ad6-496c-beeb-e10113782072 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:54.188685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6adcf5fd-0ad6-496c-beeb-e10113782072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:54.207811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:54.252759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1248/2492] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:54.261747 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff7558-b632-4802-9e4d-6c8664f33b11 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.262032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff7558-b632-4802-9e4d-6c8664f33b11 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.262309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff7558-b632-4802-9e4d-6c8664f33b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.281001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f9ff7558-b632-4802-9e4d-6c8664f33b11 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:54.282370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f9ff7558-b632-4802-9e4d-6c8664f33b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:54.290319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1245/2493] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:54.294626 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcf5fd-0ad6-496c-beeb-e10113782072 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:54.304637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcf5fd-0ad6-496c-beeb-e10113782072 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:54.309676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2baab-d3b3-49fc-9149-09c582828f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:54.310341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2baab-d3b3-49fc-9149-09c582828f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.310667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2baab-d3b3-49fc-9149-09c582828f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.310943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2baab-d3b3-49fc-9149-09c582828f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.311680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2baab-d3b3-49fc-9149-09c582828f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:14:54.311993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2baab-d3b3-49fc-9149-09c582828f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:54.313629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2baab-d3b3-49fc-9149-09c582828f7c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:54.328474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1249/2494] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:14:54.334503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff7558-b632-4802-9e4d-6c8664f33b11 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:54.343125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff7558-b632-4802-9e4d-6c8664f33b11 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:54.347912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b87da-7084-4568-89bd-5d855c9bc3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:54.348579 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b87da-7084-4568-89bd-5d855c9bc3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.348859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b87da-7084-4568-89bd-5d855c9bc3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.349165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b87da-7084-4568-89bd-5d855c9bc3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.349910 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b87da-7084-4568-89bd-5d855c9bc3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:54.351803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b87da-7084-4568-89bd-5d855c9bc3f9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:54.383285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1246/2495] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:54 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:54.384904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2baab-d3b3-49fc-9149-09c582828f7c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:54.396564 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2baab-d3b3-49fc-9149-09c582828f7c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:54.458925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9651f-65d8-42e2-87bd-7c9e66631b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:54.459608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9651f-65d8-42e2-87bd-7c9e66631b3b admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.459898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9651f-65d8-42e2-87bd-7c9e66631b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.460178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9651f-65d8-42e2-87bd-7c9e66631b3b admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.460886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9651f-65d8-42e2-87bd-7c9e66631b3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:54.462861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9651f-65d8-42e2-87bd-7c9e66631b3b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:54.483826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1250/2496] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:54 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:14:54.487964 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b87da-7084-4568-89bd-5d855c9bc3f9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:54.499046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b87da-7084-4568-89bd-5d855c9bc3f9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:54.503861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea8fd0-1e1a-4681-b59e-c409112f2280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:54.504492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea8fd0-1e1a-4681-b59e-c409112f2280 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.504767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea8fd0-1e1a-4681-b59e-c409112f2280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.505045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea8fd0-1e1a-4681-b59e-c409112f2280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.505820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea8fd0-1e1a-4681-b59e-c409112f2280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:54.507474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea8fd0-1e1a-4681-b59e-c409112f2280 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:54.522374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1247/2497] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:14:54.523532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9651f-65d8-42e2-87bd-7c9e66631b3b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:54.534846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9651f-65d8-42e2-87bd-7c9e66631b3b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:54.539654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3474-40a0-4228-a8bd-e44d52d23338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:54.540332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3474-40a0-4228-a8bd-e44d52d23338 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.540619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3474-40a0-4228-a8bd-e44d52d23338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.540902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3474-40a0-4228-a8bd-e44d52d23338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.541692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf3474-40a0-4228-a8bd-e44d52d23338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:54.543397 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf3474-40a0-4228-a8bd-e44d52d23338 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:54.557449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1251/2498] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:54.558940 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea8fd0-1e1a-4681-b59e-c409112f2280 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:54.569573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea8fd0-1e1a-4681-b59e-c409112f2280 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:54.575234 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad0cdd-3f11-456d-a17a-457d0ba56fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:54.576163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad0cdd-3f11-456d-a17a-457d0ba56fde admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.576575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad0cdd-3f11-456d-a17a-457d0ba56fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.576858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad0cdd-3f11-456d-a17a-457d0ba56fde admin admin] PATH_INFO: `/v3/projects/66c2d2a16f6f4242ae3318e99938706a/users/3adac0acc8934cbfa491ff1ac94d67b5/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.592944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad0cdd-3f11-456d-a17a-457d0ba56fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c2d2a16f6f4242ae3318e99938706a, user_id=3adac0acc8934cbfa491ff1ac94d67b5, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:54.595656 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad0cdd-3f11-456d-a17a-457d0ba56fde admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:54.611133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1248/2499] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:54 2022] PUT /identity/v3/projects/66c2d2a16f6f4242ae3318e99938706a/users/3adac0acc8934cbfa491ff1ac94d67b5/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:54.622297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595bb32-1ece-40bc-823a-c5d275a06cb2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.623001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595bb32-1ece-40bc-823a-c5d275a06cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.625223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595bb32-1ece-40bc-823a-c5d275a06cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.654213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1595bb32-1ece-40bc-823a-c5d275a06cb2 None None] MFA Rules not processed for user `3adac0acc8934cbfa491ff1ac94d67b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:54.655868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1595bb32-1ece-40bc-823a-c5d275a06cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:54.726421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1252/2500] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:54.738192 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad0cdd-3f11-456d-a17a-457d0ba56fde admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:54.739992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cdad0cdd-3f11-456d-a17a-457d0ba56fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:54.822667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad0cdd-3f11-456d-a17a-457d0ba56fde admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:54.828974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913957eb-f2b2-49c8-8cf0-381752cc2fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:54.829690 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913957eb-f2b2-49c8-8cf0-381752cc2fa1 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:54.830090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913957eb-f2b2-49c8-8cf0-381752cc2fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:54.830600 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913957eb-f2b2-49c8-8cf0-381752cc2fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:54.839894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913957eb-f2b2-49c8-8cf0-381752cc2fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:54.841566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913957eb-f2b2-49c8-8cf0-381752cc2fa1 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:54.851588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1249/2501] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:14:55.066777 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595bb32-1ece-40bc-823a-c5d275a06cb2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:55.088176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:55.133254 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595bb32-1ece-40bc-823a-c5d275a06cb2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:55.137805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:55.138660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:55.138951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:55.139230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] PATH_INFO: `/v3/users/3adac0acc8934cbfa491ff1ac94d67b5` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:55.140431 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3adac0acc8934cbfa491ff1ac94d67b5)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:55.142413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:55.192763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3adac0acc8934cbfa491ff1ac94d67b5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.205255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3adac0acc8934cbfa491ff1ac94d67b5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.212427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3adac0acc8934cbfa491ff1ac94d67b5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.213040 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3adac0acc8934cbfa491ff1ac94d67b5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.219997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3adac0acc8934cbfa491ff1ac94d67b5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.225884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1253/2502] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:55 2022] DELETE /identity/v3/users/3adac0acc8934cbfa491ff1ac94d67b5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:14:55.231129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df1e67-8fc8-4554-a4f3-8e8473d4988d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:55.231621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df1e67-8fc8-4554-a4f3-8e8473d4988d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:55.231746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df1e67-8fc8-4554-a4f3-8e8473d4988d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:55.249585 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-95df1e67-8fc8-4554-a4f3-8e8473d4988d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:55.250792 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-95df1e67-8fc8-4554-a4f3-8e8473d4988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:55.260142 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:55.312105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1250/2503] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:55.322201 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:55.323878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:55.382708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6726-52d0-4159-b70b-d21c587ffd57 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:55.386426 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b58d69-2caa-430e-baf1-d9fd3b3e27fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:55.387067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b58d69-2caa-430e-baf1-d9fd3b3e27fa service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:55.387350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b58d69-2caa-430e-baf1-d9fd3b3e27fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:55.387628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b58d69-2caa-430e-baf1-d9fd3b3e27fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:55.396879 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b58d69-2caa-430e-baf1-d9fd3b3e27fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:55.398345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b58d69-2caa-430e-baf1-d9fd3b3e27fa service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:55.408820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1254/2504] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:55.694359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df1e67-8fc8-4554-a4f3-8e8473d4988d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:55.696945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-95df1e67-8fc8-4554-a4f3-8e8473d4988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:55.719561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df1e67-8fc8-4554-a4f3-8e8473d4988d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:55.727165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:55.728688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:55.730868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:55.730868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] PATH_INFO: `/v3/projects/66c2d2a16f6f4242ae3318e99938706a` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:55.732590 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c2d2a16f6f4242ae3318e99938706a)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:55.736070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:55.788359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c2d2a16f6f4242ae3318e99938706a', 'request_id': 'req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.800494 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c2d2a16f6f4242ae3318e99938706a', 'request_id': 'req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.810312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c2d2a16f6f4242ae3318e99938706a', 'request_id': 'req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.821981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] The token cache is being invalidate because project 66c2d2a16f6f4242ae3318e99938706a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:55.822282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.822939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ec443f4c-8fbb-4dd2-b916-8a349e9b0eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:55.825449 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1251/2505] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:55 2022] DELETE /identity/v3/projects/66c2d2a16f6f4242ae3318e99938706a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:55.830100 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd449706-9532-4d62-a080-9f36495e3daa None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:55.830567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd449706-9532-4d62-a080-9f36495e3daa None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:55.830664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd449706-9532-4d62-a080-9f36495e3daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:55.849286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bd449706-9532-4d62-a080-9f36495e3daa None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:55.850562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bd449706-9532-4d62-a080-9f36495e3daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:55.894064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:55.916985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1255/2506] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:55.924829 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25182a2d-c637-4107-ab8c-2cba23b1cffc None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:55.925164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25182a2d-c637-4107-ab8c-2cba23b1cffc None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:55.925451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25182a2d-c637-4107-ab8c-2cba23b1cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:55.943978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-25182a2d-c637-4107-ab8c-2cba23b1cffc None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:55.945240 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-25182a2d-c637-4107-ab8c-2cba23b1cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:55.953679 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1252/2507] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:55.957979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd449706-9532-4d62-a080-9f36495e3daa None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:55.968216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd449706-9532-4d62-a080-9f36495e3daa None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:55.972778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbab9a7-bfcc-4301-ade8-394da879c00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:55.973464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab9a7-bfcc-4301-ade8-394da879c00a admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:55.973751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab9a7-bfcc-4301-ade8-394da879c00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:55.974026 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab9a7-bfcc-4301-ade8-394da879c00a admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:55.974781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbab9a7-bfcc-4301-ade8-394da879c00a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:14:55.975090 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbab9a7-bfcc-4301-ade8-394da879c00a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:55.976686 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbab9a7-bfcc-4301-ade8-394da879c00a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:55.989050 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1256/2508] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:14:55.993417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25182a2d-c637-4107-ab8c-2cba23b1cffc None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.003376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25182a2d-c637-4107-ab8c-2cba23b1cffc None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.008286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d2dcb-b452-4685-9923-0c072567b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.008950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d2dcb-b452-4685-9923-0c072567b280 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.009264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d2dcb-b452-4685-9923-0c072567b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.009541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d2dcb-b452-4685-9923-0c072567b280 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.010235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d2dcb-b452-4685-9923-0c072567b280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.012763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d2dcb-b452-4685-9923-0c072567b280 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.038828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1253/2509] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:55 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:56.042951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbab9a7-bfcc-4301-ade8-394da879c00a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.052935 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbab9a7-bfcc-4301-ade8-394da879c00a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.109452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aefda8-2f42-41bc-9ee7-90b945604302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.110144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aefda8-2f42-41bc-9ee7-90b945604302 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.110429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aefda8-2f42-41bc-9ee7-90b945604302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.110714 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aefda8-2f42-41bc-9ee7-90b945604302 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.111417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aefda8-2f42-41bc-9ee7-90b945604302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.113420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aefda8-2f42-41bc-9ee7-90b945604302 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.133667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1257/2510] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:56 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:56.138433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d2dcb-b452-4685-9923-0c072567b280 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.148593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d2dcb-b452-4685-9923-0c072567b280 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.153080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261c171-d7e1-4533-8277-3e5b1a1fbc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.153761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261c171-d7e1-4533-8277-3e5b1a1fbc5d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.154041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261c171-d7e1-4533-8277-3e5b1a1fbc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.154316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261c171-d7e1-4533-8277-3e5b1a1fbc5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.155033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261c171-d7e1-4533-8277-3e5b1a1fbc5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.156698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261c171-d7e1-4533-8277-3e5b1a1fbc5d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.167169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1254/2511] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:56.171160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aefda8-2f42-41bc-9ee7-90b945604302 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.181115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aefda8-2f42-41bc-9ee7-90b945604302 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.186308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b3da-7b5d-4bfa-805b-bafd74fff083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.186969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b3da-7b5d-4bfa-805b-bafd74fff083 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.187256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b3da-7b5d-4bfa-805b-bafd74fff083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.187538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b3da-7b5d-4bfa-805b-bafd74fff083 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.188266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19b3da-7b5d-4bfa-805b-bafd74fff083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.189926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19b3da-7b5d-4bfa-805b-bafd74fff083 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.201065 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1258/2512] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:56.205637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261c171-d7e1-4533-8277-3e5b1a1fbc5d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.215813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261c171-d7e1-4533-8277-3e5b1a1fbc5d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.220370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b528e8-510c-4faa-9982-eac913c9eac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.221279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b528e8-510c-4faa-9982-eac913c9eac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.221562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b528e8-510c-4faa-9982-eac913c9eac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.221841 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b528e8-510c-4faa-9982-eac913c9eac2 admin admin] PATH_INFO: `/v3/projects/9e03a0e859fe40c48ff205c146608e0e/users/b198ada2cc4f476ca493f6f15e143b24/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.236212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b528e8-510c-4faa-9982-eac913c9eac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e03a0e859fe40c48ff205c146608e0e, user_id=b198ada2cc4f476ca493f6f15e143b24, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.238905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b528e8-510c-4faa-9982-eac913c9eac2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.252610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1255/2513] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:56 2022] PUT /identity/v3/projects/9e03a0e859fe40c48ff205c146608e0e/users/b198ada2cc4f476ca493f6f15e143b24/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:56.264888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627187-ef5e-4444-8de1-51a0dc697c5a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.265213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627187-ef5e-4444-8de1-51a0dc697c5a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.265498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627187-ef5e-4444-8de1-51a0dc697c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.291783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7b627187-ef5e-4444-8de1-51a0dc697c5a None None] MFA Rules not processed for user `b198ada2cc4f476ca493f6f15e143b24`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:56.292991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7b627187-ef5e-4444-8de1-51a0dc697c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:56.357812 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1259/2514] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:56.362310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b528e8-510c-4faa-9982-eac913c9eac2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.401635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:56.425978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b528e8-510c-4faa-9982-eac913c9eac2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.430585 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7cb5-19b9-4cdc-9d8a-17618f5095e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.431238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7cb5-19b9-4cdc-9d8a-17618f5095e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.431526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7cb5-19b9-4cdc-9d8a-17618f5095e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.431807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7cb5-19b9-4cdc-9d8a-17618f5095e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.432510 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e7cb5-19b9-4cdc-9d8a-17618f5095e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.435000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e7cb5-19b9-4cdc-9d8a-17618f5095e5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.461247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1256/2515] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:56 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:56.464593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627187-ef5e-4444-8de1-51a0dc697c5a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.475213 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627187-ef5e-4444-8de1-51a0dc697c5a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.508324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:56.531601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875eec-b9f3-49cd-92a6-8850d490e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.532257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875eec-b9f3-49cd-92a6-8850d490e44d admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.532540 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875eec-b9f3-49cd-92a6-8850d490e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.532814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875eec-b9f3-49cd-92a6-8850d490e44d admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.533535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73875eec-b9f3-49cd-92a6-8850d490e44d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.535435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73875eec-b9f3-49cd-92a6-8850d490e44d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.556570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1260/2516] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:56 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:56.560244 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7cb5-19b9-4cdc-9d8a-17618f5095e5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.570736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7cb5-19b9-4cdc-9d8a-17618f5095e5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.575354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72148b7-aba5-421a-8f66-5587fafe34b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.576003 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72148b7-aba5-421a-8f66-5587fafe34b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.576290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72148b7-aba5-421a-8f66-5587fafe34b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.576567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72148b7-aba5-421a-8f66-5587fafe34b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.577317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72148b7-aba5-421a-8f66-5587fafe34b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.578950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72148b7-aba5-421a-8f66-5587fafe34b7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.590534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1257/2517] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:56.594430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875eec-b9f3-49cd-92a6-8850d490e44d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.605052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875eec-b9f3-49cd-92a6-8850d490e44d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.609708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69be4a-1695-4901-8bec-e79383a75e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.610578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69be4a-1695-4901-8bec-e79383a75e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.610871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69be4a-1695-4901-8bec-e79383a75e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.611144 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69be4a-1695-4901-8bec-e79383a75e66 admin admin] PATH_INFO: `/v3/projects/f2f7f5c9b2c24dde9c2e4d805f366bf7/users/db3fd2958d0f4111bfc9e5031da9b982/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.627024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69be4a-1695-4901-8bec-e79383a75e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f7f5c9b2c24dde9c2e4d805f366bf7, user_id=db3fd2958d0f4111bfc9e5031da9b982, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.629813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69be4a-1695-4901-8bec-e79383a75e66 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.642853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1261/2518] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:56 2022] PUT /identity/v3/projects/f2f7f5c9b2c24dde9c2e4d805f366bf7/users/db3fd2958d0f4111bfc9e5031da9b982/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:14:56.646888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72148b7-aba5-421a-8f66-5587fafe34b7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.657196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72148b7-aba5-421a-8f66-5587fafe34b7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.714008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea30d7-8fbb-4b17-9604-b96420b4f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.714677 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea30d7-8fbb-4b17-9604-b96420b4f8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.714966 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea30d7-8fbb-4b17-9604-b96420b4f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.715249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea30d7-8fbb-4b17-9604-b96420b4f8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.715995 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea30d7-8fbb-4b17-9604-b96420b4f8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.717613 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea30d7-8fbb-4b17-9604-b96420b4f8e2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.727291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1258/2519] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:56.731965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69be4a-1695-4901-8bec-e79383a75e66 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.742350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69be4a-1695-4901-8bec-e79383a75e66 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.747104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2a55-e678-457c-8e5b-0bca476dd73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.747977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2a55-e678-457c-8e5b-0bca476dd73b admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.748268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2a55-e678-457c-8e5b-0bca476dd73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.748550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2a55-e678-457c-8e5b-0bca476dd73b admin admin] PATH_INFO: `/v3/projects/f2f7f5c9b2c24dde9c2e4d805f366bf7/users/db3fd2958d0f4111bfc9e5031da9b982/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.750644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a2a55-e678-457c-8e5b-0bca476dd73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f7f5c9b2c24dde9c2e4d805f366bf7, user_id=db3fd2958d0f4111bfc9e5031da9b982, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.753318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a2a55-e678-457c-8e5b-0bca476dd73b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.764166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1262/2520] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:56 2022] PUT /identity/v3/projects/f2f7f5c9b2c24dde9c2e4d805f366bf7/users/db3fd2958d0f4111bfc9e5031da9b982/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:56.774599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b420-c327-4381-a485-75a354d1cc4e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.774901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b420-c327-4381-a485-75a354d1cc4e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.775166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b420-c327-4381-a485-75a354d1cc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.801435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-44c0b420-c327-4381-a485-75a354d1cc4e None None] MFA Rules not processed for user `db3fd2958d0f4111bfc9e5031da9b982`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:56.802637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-44c0b420-c327-4381-a485-75a354d1cc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:56.869329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1259/2521] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:56.880750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2a55-e678-457c-8e5b-0bca476dd73b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:56.882549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8e0a2a55-e678-457c-8e5b-0bca476dd73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:56.946619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2a55-e678-457c-8e5b-0bca476dd73b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:56.951257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039049d-1042-4d1e-a34b-ae8f25820378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:56.952024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039049d-1042-4d1e-a34b-ae8f25820378 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:56.952177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039049d-1042-4d1e-a34b-ae8f25820378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:56.952458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039049d-1042-4d1e-a34b-ae8f25820378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:56.962720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039049d-1042-4d1e-a34b-ae8f25820378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:56.964198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039049d-1042-4d1e-a34b-ae8f25820378 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:56.974447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1263/2522] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:57.356605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0b420-c327-4381-a485-75a354d1cc4e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:57.390228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:57.421681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0b420-c327-4381-a485-75a354d1cc4e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:57.426191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:57.427055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:57.427344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:57.427625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] PATH_INFO: `/v3/users/b198ada2cc4f476ca493f6f15e143b24` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:57.428818 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b198ada2cc4f476ca493f6f15e143b24)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:57.430795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:57.476760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b198ada2cc4f476ca493f6f15e143b24'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:57.489300 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b198ada2cc4f476ca493f6f15e143b24'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:57.497219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b198ada2cc4f476ca493f6f15e143b24'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:57.504224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b198ada2cc4f476ca493f6f15e143b24'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:57.504854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b198ada2cc4f476ca493f6f15e143b24'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:57.510803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1260/2523] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:57 2022] DELETE /identity/v3/users/b198ada2cc4f476ca493f6f15e143b24 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:57.515306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24fb43-0674-40e7-8921-0675441ee09c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:57.515597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24fb43-0674-40e7-8921-0675441ee09c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:57.515875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24fb43-0674-40e7-8921-0675441ee09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:57.534853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-af24fb43-0674-40e7-8921-0675441ee09c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:57.536059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-af24fb43-0674-40e7-8921-0675441ee09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:57.599258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1264/2524] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:57.609432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:57.610903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:57.672212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f9ef-0d56-4a4f-9e95-9051e9a7b171 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:57.675802 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fe038-fdee-4c8d-bf7b-da2191851396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:57.676458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fe038-fdee-4c8d-bf7b-da2191851396 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:57.676745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fe038-fdee-4c8d-bf7b-da2191851396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:57.677036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fe038-fdee-4c8d-bf7b-da2191851396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:57.686936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fe038-fdee-4c8d-bf7b-da2191851396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:57.688391 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fe038-fdee-4c8d-bf7b-da2191851396 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:57.698777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1261/2525] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:57.980961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24fb43-0674-40e7-8921-0675441ee09c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:57.982780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-af24fb43-0674-40e7-8921-0675441ee09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:57.995861 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24fb43-0674-40e7-8921-0675441ee09c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:58.002229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:58.003098 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:58.003385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:58.003673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] PATH_INFO: `/v3/projects/9e03a0e859fe40c48ff205c146608e0e` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:58.004877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e03a0e859fe40c48ff205c146608e0e)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:58.006823 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:58.039174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:58.055663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e03a0e859fe40c48ff205c146608e0e', 'request_id': 'req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.062406 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e03a0e859fe40c48ff205c146608e0e', 'request_id': 'req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.069413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e03a0e859fe40c48ff205c146608e0e', 'request_id': 'req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.080106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] The token cache is being invalidate because project 9e03a0e859fe40c48ff205c146608e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:58.080396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.081031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.082596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1265/2526] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:57 2022] DELETE /identity/v3/projects/9e03a0e859fe40c48ff205c146608e0e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:58.086568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fe038-fdee-4c8d-bf7b-da2191851396 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:58.096861 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fe038-fdee-4c8d-bf7b-da2191851396 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:58.143734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:58.155714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:58.156575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:58.156865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:58.157185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] PATH_INFO: `/v3/users/db3fd2958d0f4111bfc9e5031da9b982` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:58.158503 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3fd2958d0f4111bfc9e5031da9b982)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:58.160325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:58.209732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3fd2958d0f4111bfc9e5031da9b982'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.222224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3fd2958d0f4111bfc9e5031da9b982'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.229836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3fd2958d0f4111bfc9e5031da9b982'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.237806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3fd2958d0f4111bfc9e5031da9b982'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.238618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65aa940c-03fe-49e3-8377-69d9660707b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3fd2958d0f4111bfc9e5031da9b982'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.245167 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1262/2527] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:58 2022] DELETE /identity/v3/users/db3fd2958d0f4111bfc9e5031da9b982 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:58.522074 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:58.523515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:58.536771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8e3ae-5445-4adc-bb5b-0c92a398fc36 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:58.594816 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:58.595677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:58.595958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:58.596233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] PATH_INFO: `/v3/projects/f2f7f5c9b2c24dde9c2e4d805f366bf7` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:58.597462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f7f5c9b2c24dde9c2e4d805f366bf7)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:58.599359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:58.646461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f7f5c9b2c24dde9c2e4d805f366bf7', 'request_id': 'req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.652875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f7f5c9b2c24dde9c2e4d805f366bf7', 'request_id': 'req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.660285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f7f5c9b2c24dde9c2e4d805f366bf7', 'request_id': 'req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.670828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] The token cache is being invalidate because project f2f7f5c9b2c24dde9c2e4d805f366bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:14:58.671109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.671732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c7ee1caa-1d1e-4dca-b913-c1d22b8c4bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:58.673294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1266/2528] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:14:58 2022] DELETE /identity/v3/projects/f2f7f5c9b2c24dde9c2e4d805f366bf7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:58.678146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17136-70af-4166-8163-7c46e328050b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:58.678623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17136-70af-4166-8163-7c46e328050b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:58.678753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17136-70af-4166-8163-7c46e328050b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:58.697909 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-74d17136-70af-4166-8163-7c46e328050b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:58.699755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-74d17136-70af-4166-8163-7c46e328050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:58.758919 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1263/2529] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:58.766618 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef889b-8872-46ee-bb81-939d7f99f8a8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:58.766929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef889b-8872-46ee-bb81-939d7f99f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:58.767207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef889b-8872-46ee-bb81-939d7f99f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:58.786781 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a0ef889b-8872-46ee-bb81-939d7f99f8a8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:58.787995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a0ef889b-8872-46ee-bb81-939d7f99f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:58.796530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1267/2530] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:58.802197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d17136-70af-4166-8163-7c46e328050b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:58.812948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d17136-70af-4166-8163-7c46e328050b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:58.817625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cb9e5-95c5-48d4-b04e-648b057b17bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:58.818538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cb9e5-95c5-48d4-b04e-648b057b17bd admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:58.818823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cb9e5-95c5-48d4-b04e-648b057b17bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:58.819101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cb9e5-95c5-48d4-b04e-648b057b17bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:58.819835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cb9e5-95c5-48d4-b04e-648b057b17bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:14:58.820147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cb9e5-95c5-48d4-b04e-648b057b17bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:58.821794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cb9e5-95c5-48d4-b04e-648b057b17bd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:58.837632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1264/2531] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:14:58.840695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef889b-8872-46ee-bb81-939d7f99f8a8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:58.854332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef889b-8872-46ee-bb81-939d7f99f8a8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:58.859524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaaad41-d90c-4708-b8ad-b9ea7f51e5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:58.860187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaaad41-d90c-4708-b8ad-b9ea7f51e5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:58.860471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaaad41-d90c-4708-b8ad-b9ea7f51e5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:58.860748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaaad41-d90c-4708-b8ad-b9ea7f51e5e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:58.861503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaaad41-d90c-4708-b8ad-b9ea7f51e5e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:58.863414 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaaad41-d90c-4708-b8ad-b9ea7f51e5e4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:58.890846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1268/2532] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:58 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:58.895115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cb9e5-95c5-48d4-b04e-648b057b17bd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:58.906402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cb9e5-95c5-48d4-b04e-648b057b17bd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:58.964651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75389ba5-a783-49d5-b18b-21b3e615b8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:58.965376 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75389ba5-a783-49d5-b18b-21b3e615b8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:58.965749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75389ba5-a783-49d5-b18b-21b3e615b8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:58.966032 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75389ba5-a783-49d5-b18b-21b3e615b8b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:58.966727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75389ba5-a783-49d5-b18b-21b3e615b8b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:58.968706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75389ba5-a783-49d5-b18b-21b3e615b8b4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:58.975412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:58.989929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1265/2533] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:58 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:58.993969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaaad41-d90c-4708-b8ad-b9ea7f51e5e4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.004788 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaaad41-d90c-4708-b8ad-b9ea7f51e5e4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.009499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d44edc-b954-4649-aac3-f8185c068ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.010086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d44edc-b954-4649-aac3-f8185c068ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.010371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d44edc-b954-4649-aac3-f8185c068ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.010678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d44edc-b954-4649-aac3-f8185c068ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.011459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d44edc-b954-4649-aac3-f8185c068ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.013043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d44edc-b954-4649-aac3-f8185c068ff4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.023212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1269/2534] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:59.028928 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75389ba5-a783-49d5-b18b-21b3e615b8b4 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.039257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75389ba5-a783-49d5-b18b-21b3e615b8b4 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.044436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec390c4-3b0c-4f15-b64b-7a2b23ce65b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.045110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec390c4-3b0c-4f15-b64b-7a2b23ce65b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.045413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec390c4-3b0c-4f15-b64b-7a2b23ce65b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.045698 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec390c4-3b0c-4f15-b64b-7a2b23ce65b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.046422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec390c4-3b0c-4f15-b64b-7a2b23ce65b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.048022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec390c4-3b0c-4f15-b64b-7a2b23ce65b0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.060824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1266/2535] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:59.064927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d44edc-b954-4649-aac3-f8185c068ff4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.075187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d44edc-b954-4649-aac3-f8185c068ff4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.079961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f83e96-286c-4e58-bfba-ee033ff9e795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.080828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f83e96-286c-4e58-bfba-ee033ff9e795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.081139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f83e96-286c-4e58-bfba-ee033ff9e795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.081421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f83e96-286c-4e58-bfba-ee033ff9e795 admin admin] PATH_INFO: `/v3/projects/a8f8355c0e7e4eeb861d96bd4fedc410/users/9fdd59f3a79d4fbaa9a9734ec603dac4/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.083695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:59.097877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f83e96-286c-4e58-bfba-ee033ff9e795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f8355c0e7e4eeb861d96bd4fedc410, user_id=9fdd59f3a79d4fbaa9a9734ec603dac4, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.100503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f83e96-286c-4e58-bfba-ee033ff9e795 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.113613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1270/2536] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:59 2022] PUT /identity/v3/projects/a8f8355c0e7e4eeb861d96bd4fedc410/users/9fdd59f3a79d4fbaa9a9734ec603dac4/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:59.124319 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f95da1-c63a-4819-ade2-22dd6022cc04 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.124620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f95da1-c63a-4819-ade2-22dd6022cc04 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.124905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f95da1-c63a-4819-ade2-22dd6022cc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.150793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d9f95da1-c63a-4819-ade2-22dd6022cc04 None None] MFA Rules not processed for user `9fdd59f3a79d4fbaa9a9734ec603dac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:59.152620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d9f95da1-c63a-4819-ade2-22dd6022cc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:59.218416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1267/2537] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:59.222843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f83e96-286c-4e58-bfba-ee033ff9e795 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.286465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f83e96-286c-4e58-bfba-ee033ff9e795 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.291028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85830176-3b2e-4d9a-9e7f-c513b345e84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.291676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85830176-3b2e-4d9a-9e7f-c513b345e84e admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.291975 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85830176-3b2e-4d9a-9e7f-c513b345e84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.292995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85830176-3b2e-4d9a-9e7f-c513b345e84e admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.293738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85830176-3b2e-4d9a-9e7f-c513b345e84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.295637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85830176-3b2e-4d9a-9e7f-c513b345e84e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.322526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1271/2538] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:14:59 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:14:59.326619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f95da1-c63a-4819-ade2-22dd6022cc04 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.337759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f95da1-c63a-4819-ade2-22dd6022cc04 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.394487 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c053d-4175-4045-99c0-08a7f058a000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.395156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c053d-4175-4045-99c0-08a7f058a000 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.395495 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c053d-4175-4045-99c0-08a7f058a000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.395780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c053d-4175-4045-99c0-08a7f058a000 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.396470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c053d-4175-4045-99c0-08a7f058a000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.398413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c053d-4175-4045-99c0-08a7f058a000 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.419196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1268/2539] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:14:59 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:14:59.422987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85830176-3b2e-4d9a-9e7f-c513b345e84e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.433458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85830176-3b2e-4d9a-9e7f-c513b345e84e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.438125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb2e49-5ae4-4e19-9447-9c5fc2e71f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.438779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb2e49-5ae4-4e19-9447-9c5fc2e71f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.439071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb2e49-5ae4-4e19-9447-9c5fc2e71f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.439348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb2e49-5ae4-4e19-9447-9c5fc2e71f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.440103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb2e49-5ae4-4e19-9447-9c5fc2e71f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.441738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb2e49-5ae4-4e19-9447-9c5fc2e71f31 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.452715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1272/2540] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:59.456759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c053d-4175-4045-99c0-08a7f058a000 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.467442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c053d-4175-4045-99c0-08a7f058a000 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.472000 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520b072-f4c2-4165-8aef-b30596300bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.472871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520b072-f4c2-4165-8aef-b30596300bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.473186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520b072-f4c2-4165-8aef-b30596300bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.473465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520b072-f4c2-4165-8aef-b30596300bfe admin admin] PATH_INFO: `/v3/projects/9a060a082bec48c5a7eb03391a7a85b2/users/bad68de079884bf8b1e0bb7874f635db/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.488501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520b072-f4c2-4165-8aef-b30596300bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a060a082bec48c5a7eb03391a7a85b2, user_id=bad68de079884bf8b1e0bb7874f635db, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.491238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520b072-f4c2-4165-8aef-b30596300bfe admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.504843 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1269/2541] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:59 2022] PUT /identity/v3/projects/9a060a082bec48c5a7eb03391a7a85b2/users/bad68de079884bf8b1e0bb7874f635db/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:59.508739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb2e49-5ae4-4e19-9447-9c5fc2e71f31 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.519693 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb2e49-5ae4-4e19-9447-9c5fc2e71f31 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.575724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8070a7d-d35b-4365-b5e3-2ca53cad2abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.576309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8070a7d-d35b-4365-b5e3-2ca53cad2abe admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.576594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8070a7d-d35b-4365-b5e3-2ca53cad2abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.576871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8070a7d-d35b-4365-b5e3-2ca53cad2abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.577622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8070a7d-d35b-4365-b5e3-2ca53cad2abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.579203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8070a7d-d35b-4365-b5e3-2ca53cad2abe admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.589902 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1273/2542] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:14:59.594020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520b072-f4c2-4165-8aef-b30596300bfe admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.604432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520b072-f4c2-4165-8aef-b30596300bfe admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.609077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86913fe0-bfa3-4c1f-ad69-332dc6166354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.609980 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86913fe0-bfa3-4c1f-ad69-332dc6166354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.610264 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86913fe0-bfa3-4c1f-ad69-332dc6166354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.610540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86913fe0-bfa3-4c1f-ad69-332dc6166354 admin admin] PATH_INFO: `/v3/projects/9a060a082bec48c5a7eb03391a7a85b2/users/bad68de079884bf8b1e0bb7874f635db/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.612553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86913fe0-bfa3-4c1f-ad69-332dc6166354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a060a082bec48c5a7eb03391a7a85b2, user_id=bad68de079884bf8b1e0bb7874f635db, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.615213 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86913fe0-bfa3-4c1f-ad69-332dc6166354 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.626530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1270/2543] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:14:59 2022] PUT /identity/v3/projects/9a060a082bec48c5a7eb03391a7a85b2/users/bad68de079884bf8b1e0bb7874f635db/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:14:59.637071 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9bac3-3cbd-4d0a-bbd0-e3ce9f5f27d5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.637400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9bac3-3cbd-4d0a-bbd0-e3ce9f5f27d5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.637687 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9bac3-3cbd-4d0a-bbd0-e3ce9f5f27d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.664139 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-73b9bac3-3cbd-4d0a-bbd0-e3ce9f5f27d5 None None] MFA Rules not processed for user `bad68de079884bf8b1e0bb7874f635db`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:14:59.665516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-73b9bac3-3cbd-4d0a-bbd0-e3ce9f5f27d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:59.706927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:59.735064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1274/2544] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:14:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:14:59.746212 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86913fe0-bfa3-4c1f-ad69-332dc6166354 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.747960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-86913fe0-bfa3-4c1f-ad69-332dc6166354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:14:59.770716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:14:59.807077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86913fe0-bfa3-4c1f-ad69-332dc6166354 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.810777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff810671-c67b-42cc-acf4-16f3abe4219f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.811415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff810671-c67b-42cc-acf4-16f3abe4219f service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.811704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff810671-c67b-42cc-acf4-16f3abe4219f service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.811979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff810671-c67b-42cc-acf4-16f3abe4219f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.821193 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff810671-c67b-42cc-acf4-16f3abe4219f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.822612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff810671-c67b-42cc-acf4-16f3abe4219f service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.832429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1271/2545] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:14:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:14:59.860346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9bac3-3cbd-4d0a-bbd0-e3ce9f5f27d5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:14:59.921200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9bac3-3cbd-4d0a-bbd0-e3ce9f5f27d5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:14:59.925799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:14:59.926651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:14:59.926937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:14:59.927215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] PATH_INFO: `/v3/users/9fdd59f3a79d4fbaa9a9734ec603dac4` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:14:59.928425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fdd59f3a79d4fbaa9a9734ec603dac4)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:14:59.930342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:14:59.981952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fdd59f3a79d4fbaa9a9734ec603dac4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:14:59.996986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fdd59f3a79d4fbaa9a9734ec603dac4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.005103 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fdd59f3a79d4fbaa9a9734ec603dac4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.005733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fdd59f3a79d4fbaa9a9734ec603dac4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.013584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fdd59f3a79d4fbaa9a9734ec603dac4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.019613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1275/2546] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:14:59 2022] DELETE /identity/v3/users/9fdd59f3a79d4fbaa9a9734ec603dac4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:00.024221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9271f4-f21c-4db1-b782-db6d58e6efbf None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:00.024529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9271f4-f21c-4db1-b782-db6d58e6efbf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:00.024808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9271f4-f21c-4db1-b782-db6d58e6efbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:00.042476 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-de9271f4-f21c-4db1-b782-db6d58e6efbf None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:00.043667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-de9271f4-f21c-4db1-b782-db6d58e6efbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:00.103561 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1272/2547] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:00.113115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:00.114525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:00.173503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4546d5c-1779-4216-95ca-268cc1011e7a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:00.177385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46661bd8-a4f8-4a8e-825d-a159db289561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:00.178033 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46661bd8-a4f8-4a8e-825d-a159db289561 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:00.178320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46661bd8-a4f8-4a8e-825d-a159db289561 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:00.178600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46661bd8-a4f8-4a8e-825d-a159db289561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:00.188669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46661bd8-a4f8-4a8e-825d-a159db289561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:00.190143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46661bd8-a4f8-4a8e-825d-a159db289561 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:00.195498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:00.201402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1276/2548] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:00.477509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9271f4-f21c-4db1-b782-db6d58e6efbf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:00.478961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-de9271f4-f21c-4db1-b782-db6d58e6efbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:00.491458 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9271f4-f21c-4db1-b782-db6d58e6efbf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:00.496726 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:00.497642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:00.497931 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:00.498214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] PATH_INFO: `/v3/projects/a8f8355c0e7e4eeb861d96bd4fedc410` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:00.499420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f8355c0e7e4eeb861d96bd4fedc410)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:00.501388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:00.547691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f8355c0e7e4eeb861d96bd4fedc410', 'request_id': 'req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.553866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f8355c0e7e4eeb861d96bd4fedc410', 'request_id': 'req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.562386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f8355c0e7e4eeb861d96bd4fedc410', 'request_id': 'req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.573341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] The token cache is being invalidate because project a8f8355c0e7e4eeb861d96bd4fedc410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:00.573617 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.574249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.575795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1273/2549] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:00 2022] DELETE /identity/v3/projects/a8f8355c0e7e4eeb861d96bd4fedc410 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:00.579779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46661bd8-a4f8-4a8e-825d-a159db289561 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:00.589552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46661bd8-a4f8-4a8e-825d-a159db289561 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:00.648853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:00.649658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:00.649944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:00.650221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] PATH_INFO: `/v3/users/bad68de079884bf8b1e0bb7874f635db` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:00.651945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad68de079884bf8b1e0bb7874f635db)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:00.653971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:00.700036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad68de079884bf8b1e0bb7874f635db'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.711904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad68de079884bf8b1e0bb7874f635db'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.719759 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad68de079884bf8b1e0bb7874f635db'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.720383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad68de079884bf8b1e0bb7874f635db'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.727372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8e985315-2068-4634-bdb7-5767b68bbe2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad68de079884bf8b1e0bb7874f635db'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:00.733803 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1277/2550] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:00 2022] DELETE /identity/v3/users/bad68de079884bf8b1e0bb7874f635db => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:00.999573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.001060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:01.014677 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3dd57-4ff3-44cc-b58a-9e3354beb6b1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.058917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:01.074796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.075716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.075989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.076272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] PATH_INFO: `/v3/projects/9a060a082bec48c5a7eb03391a7a85b2` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.077499 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a060a082bec48c5a7eb03391a7a85b2)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.079420 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.126405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a060a082bec48c5a7eb03391a7a85b2', 'request_id': 'req-771a0f9d-5e97-4673-8c90-48ffb694c4c8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:01.132493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a060a082bec48c5a7eb03391a7a85b2', 'request_id': 'req-771a0f9d-5e97-4673-8c90-48ffb694c4c8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:01.139984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a060a082bec48c5a7eb03391a7a85b2', 'request_id': 'req-771a0f9d-5e97-4673-8c90-48ffb694c4c8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:01.150421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] The token cache is being invalidate because project 9a060a082bec48c5a7eb03391a7a85b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:01.150711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:01.151986 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-771a0f9d-5e97-4673-8c90-48ffb694c4c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:01.153576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1274/2551] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:00 2022] DELETE /identity/v3/projects/9a060a082bec48c5a7eb03391a7a85b2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:01.158516 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5aae0-423d-4210-a479-f661823c0b3f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.158992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5aae0-423d-4210-a479-f661823c0b3f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.159112 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5aae0-423d-4210-a479-f661823c0b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.176829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c3c5aae0-423d-4210-a479-f661823c0b3f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:01.178007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c3c5aae0-423d-4210-a479-f661823c0b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:01.237815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1278/2552] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:01.246794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c43733-97ca-4190-94c7-284c016137e8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.247084 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c43733-97ca-4190-94c7-284c016137e8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.247393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c43733-97ca-4190-94c7-284c016137e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.266770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-01c43733-97ca-4190-94c7-284c016137e8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:01.267977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-01c43733-97ca-4190-94c7-284c016137e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:01.275855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1275/2553] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:01.280693 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5aae0-423d-4210-a479-f661823c0b3f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.290389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5aae0-423d-4210-a479-f661823c0b3f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.294939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6df6b-b876-490f-b12c-2a253f403998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.295584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6df6b-b876-490f-b12c-2a253f403998 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.295871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6df6b-b876-490f-b12c-2a253f403998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.296151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6df6b-b876-490f-b12c-2a253f403998 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.296877 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6df6b-b876-490f-b12c-2a253f403998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:15:01.297218 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6df6b-b876-490f-b12c-2a253f403998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.298814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6df6b-b876-490f-b12c-2a253f403998 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.310347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1279/2554] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:15:01.314554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c43733-97ca-4190-94c7-284c016137e8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.325313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c43733-97ca-4190-94c7-284c016137e8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.329896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee5eea-a5c4-4134-8028-ff951713b532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.330555 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee5eea-a5c4-4134-8028-ff951713b532 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.330843 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee5eea-a5c4-4134-8028-ff951713b532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.331128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee5eea-a5c4-4134-8028-ff951713b532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.331838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee5eea-a5c4-4134-8028-ff951713b532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.333808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee5eea-a5c4-4134-8028-ff951713b532 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.360069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1276/2555] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:01 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:01.364289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6df6b-b876-490f-b12c-2a253f403998 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.374958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6df6b-b876-490f-b12c-2a253f403998 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.378497 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:01.435766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57879b1f-0f62-4f6d-bf9a-8aab10f742a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.436433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57879b1f-0f62-4f6d-bf9a-8aab10f742a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.436716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57879b1f-0f62-4f6d-bf9a-8aab10f742a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.436995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57879b1f-0f62-4f6d-bf9a-8aab10f742a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.437732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57879b1f-0f62-4f6d-bf9a-8aab10f742a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.439650 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57879b1f-0f62-4f6d-bf9a-8aab10f742a2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.459094 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1280/2556] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:01 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:01.463344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee5eea-a5c4-4134-8028-ff951713b532 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.474205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee5eea-a5c4-4134-8028-ff951713b532 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.478889 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c01cf-f01a-45e3-9b07-4d4ccf29a93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.480216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c01cf-f01a-45e3-9b07-4d4ccf29a93e admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.480511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c01cf-f01a-45e3-9b07-4d4ccf29a93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.480802 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c01cf-f01a-45e3-9b07-4d4ccf29a93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.481556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c01cf-f01a-45e3-9b07-4d4ccf29a93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.483146 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c01cf-f01a-45e3-9b07-4d4ccf29a93e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.494679 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1277/2557] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:01.498311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57879b1f-0f62-4f6d-bf9a-8aab10f742a2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.508453 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57879b1f-0f62-4f6d-bf9a-8aab10f742a2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.513182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5f0ab-28af-472d-9338-490c7d06e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.513832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5f0ab-28af-472d-9338-490c7d06e50f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.514121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5f0ab-28af-472d-9338-490c7d06e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.514407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5f0ab-28af-472d-9338-490c7d06e50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.515152 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5f0ab-28af-472d-9338-490c7d06e50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.516745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5f0ab-28af-472d-9338-490c7d06e50f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.527143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1281/2558] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:01.531051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c01cf-f01a-45e3-9b07-4d4ccf29a93e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.541282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c01cf-f01a-45e3-9b07-4d4ccf29a93e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.546626 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d637c4-1c04-4b13-ac02-1b75c8b56afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.547528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d637c4-1c04-4b13-ac02-1b75c8b56afa admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.547810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d637c4-1c04-4b13-ac02-1b75c8b56afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.548088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d637c4-1c04-4b13-ac02-1b75c8b56afa admin admin] PATH_INFO: `/v3/projects/60b17c22c299464cb240eebfb4f4dd5d/users/381e4a68ab70487ba1ae2511dc867bf2/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.562550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d637c4-1c04-4b13-ac02-1b75c8b56afa admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b17c22c299464cb240eebfb4f4dd5d, user_id=381e4a68ab70487ba1ae2511dc867bf2, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.565199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d637c4-1c04-4b13-ac02-1b75c8b56afa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.578383 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1278/2559] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:01 2022] PUT /identity/v3/projects/60b17c22c299464cb240eebfb4f4dd5d/users/381e4a68ab70487ba1ae2511dc867bf2/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:01.588437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95736a-6947-4ff5-9f8b-a9be249f02d4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.589274 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95736a-6947-4ff5-9f8b-a9be249f02d4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.589585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95736a-6947-4ff5-9f8b-a9be249f02d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.617268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9e95736a-6947-4ff5-9f8b-a9be249f02d4 None None] MFA Rules not processed for user `381e4a68ab70487ba1ae2511dc867bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:01.618694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9e95736a-6947-4ff5-9f8b-a9be249f02d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:01.684319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1282/2560] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:01.688868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d637c4-1c04-4b13-ac02-1b75c8b56afa admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.749552 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d637c4-1c04-4b13-ac02-1b75c8b56afa admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.754228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37cc52-3820-4323-849c-54c8b2c4b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.754873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37cc52-3820-4323-849c-54c8b2c4b299 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.755158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37cc52-3820-4323-849c-54c8b2c4b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.755434 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37cc52-3820-4323-849c-54c8b2c4b299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.756115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37cc52-3820-4323-849c-54c8b2c4b299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.758068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37cc52-3820-4323-849c-54c8b2c4b299 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.777151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:01.785644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1279/2561] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:01 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:01.789629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95736a-6947-4ff5-9f8b-a9be249f02d4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.800372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95736a-6947-4ff5-9f8b-a9be249f02d4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.855761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c69b82-3c39-4dd7-9973-5c59b9becd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.856430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c69b82-3c39-4dd7-9973-5c59b9becd3a admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.856716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c69b82-3c39-4dd7-9973-5c59b9becd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.856997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c69b82-3c39-4dd7-9973-5c59b9becd3a admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.857746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c69b82-3c39-4dd7-9973-5c59b9becd3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.859684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c69b82-3c39-4dd7-9973-5c59b9becd3a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.882113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1283/2562] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:01 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:15:01.883326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37cc52-3820-4323-849c-54c8b2c4b299 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.894447 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37cc52-3820-4323-849c-54c8b2c4b299 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.899096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cb252-95a3-46a9-895d-2a727094c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.899753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cb252-95a3-46a9-895d-2a727094c296 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.900039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cb252-95a3-46a9-895d-2a727094c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.900338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cb252-95a3-46a9-895d-2a727094c296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.901057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cb252-95a3-46a9-895d-2a727094c296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.903356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cb252-95a3-46a9-895d-2a727094c296 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.914035 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1280/2563] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:15:01.918156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c69b82-3c39-4dd7-9973-5c59b9becd3a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.924943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:01.928549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c69b82-3c39-4dd7-9973-5c59b9becd3a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.934495 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210f749-e00e-4c70-ad9c-f1fa5a4f6578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.935154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210f749-e00e-4c70-ad9c-f1fa5a4f6578 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.935443 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210f749-e00e-4c70-ad9c-f1fa5a4f6578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.935730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210f749-e00e-4c70-ad9c-f1fa5a4f6578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.936462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210f749-e00e-4c70-ad9c-f1fa5a4f6578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.938080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210f749-e00e-4c70-ad9c-f1fa5a4f6578 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:01.949051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1284/2564] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:01.953281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cb252-95a3-46a9-895d-2a727094c296 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:01.963125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cb252-95a3-46a9-895d-2a727094c296 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:01.968697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63828616-1516-4103-abce-93a9aa3b5d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:01.969593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63828616-1516-4103-abce-93a9aa3b5d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:01.969872 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63828616-1516-4103-abce-93a9aa3b5d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:01.970143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63828616-1516-4103-abce-93a9aa3b5d70 admin admin] PATH_INFO: `/v3/projects/f7df03587396443a902f48c65da5a010/users/128232763b3c420c83ae62a3052c4ca9/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:01.985026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63828616-1516-4103-abce-93a9aa3b5d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7df03587396443a902f48c65da5a010, user_id=128232763b3c420c83ae62a3052c4ca9, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:01.987693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63828616-1516-4103-abce-93a9aa3b5d70 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:02.001083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1281/2565] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:01 2022] PUT /identity/v3/projects/f7df03587396443a902f48c65da5a010/users/128232763b3c420c83ae62a3052c4ca9/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:02.011898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d569bd-7d91-43b2-a4f4-decad38288c8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.012192 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d569bd-7d91-43b2-a4f4-decad38288c8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.012487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d569bd-7d91-43b2-a4f4-decad38288c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.040490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-69d569bd-7d91-43b2-a4f4-decad38288c8 None None] MFA Rules not processed for user `128232763b3c420c83ae62a3052c4ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:02.041762 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-69d569bd-7d91-43b2-a4f4-decad38288c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:02.108175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1285/2566] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:02.113865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63828616-1516-4103-abce-93a9aa3b5d70 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:02.176079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63828616-1516-4103-abce-93a9aa3b5d70 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:02.180723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a515d0b-1b66-4b36-ab4f-9c290e5e4da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:02.181418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a515d0b-1b66-4b36-ab4f-9c290e5e4da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.181706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a515d0b-1b66-4b36-ab4f-9c290e5e4da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.181987 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a515d0b-1b66-4b36-ab4f-9c290e5e4da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.183179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a515d0b-1b66-4b36-ab4f-9c290e5e4da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:02.185116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a515d0b-1b66-4b36-ab4f-9c290e5e4da3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:02.211028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1282/2567] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:02 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:02.215015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d569bd-7d91-43b2-a4f4-decad38288c8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:02.225807 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d569bd-7d91-43b2-a4f4-decad38288c8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:02.284606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e7fe-a6a4-4fb1-8ed3-41cc3f6ee758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:02.285303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e7fe-a6a4-4fb1-8ed3-41cc3f6ee758 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.285590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e7fe-a6a4-4fb1-8ed3-41cc3f6ee758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.285870 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e7fe-a6a4-4fb1-8ed3-41cc3f6ee758 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.286577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3e7fe-a6a4-4fb1-8ed3-41cc3f6ee758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:02.288888 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3e7fe-a6a4-4fb1-8ed3-41cc3f6ee758 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:02.312712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1286/2568] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:02 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:02.314292 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a515d0b-1b66-4b36-ab4f-9c290e5e4da3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:02.327708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a515d0b-1b66-4b36-ab4f-9c290e5e4da3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:02.332731 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1760e5-469e-44e7-93ca-fa76a18a75d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:02.333497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1760e5-469e-44e7-93ca-fa76a18a75d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.333782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1760e5-469e-44e7-93ca-fa76a18a75d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.334069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1760e5-469e-44e7-93ca-fa76a18a75d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.334790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1760e5-469e-44e7-93ca-fa76a18a75d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:02.336398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1760e5-469e-44e7-93ca-fa76a18a75d0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:02.350521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1283/2569] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:15:02.354143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e7fe-a6a4-4fb1-8ed3-41cc3f6ee758 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:02.364396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e7fe-a6a4-4fb1-8ed3-41cc3f6ee758 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:02.373663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0215a-0a5a-45f7-b528-1c08f3c8a751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:02.374553 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0215a-0a5a-45f7-b528-1c08f3c8a751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.374867 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0215a-0a5a-45f7-b528-1c08f3c8a751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.375147 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0215a-0a5a-45f7-b528-1c08f3c8a751 admin admin] PATH_INFO: `/v3/projects/004e714b57b844bea3748a8a633e7892/users/cbb768a3cee54f98ab1c2c27ff39cc0d/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.391981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0215a-0a5a-45f7-b528-1c08f3c8a751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=004e714b57b844bea3748a8a633e7892, user_id=cbb768a3cee54f98ab1c2c27ff39cc0d, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:02.394708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0215a-0a5a-45f7-b528-1c08f3c8a751 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:02.408451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1287/2570] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:02 2022] PUT /identity/v3/projects/004e714b57b844bea3748a8a633e7892/users/cbb768a3cee54f98ab1c2c27ff39cc0d/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:02.412650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1760e5-469e-44e7-93ca-fa76a18a75d0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:02.423640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1760e5-469e-44e7-93ca-fa76a18a75d0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:02.487805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd091b34-f3df-48da-8039-db795505d784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:02.488399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091b34-f3df-48da-8039-db795505d784 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.488688 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091b34-f3df-48da-8039-db795505d784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.488970 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091b34-f3df-48da-8039-db795505d784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.489764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd091b34-f3df-48da-8039-db795505d784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:02.491373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd091b34-f3df-48da-8039-db795505d784 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:02.501819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1284/2571] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:02.506020 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0215a-0a5a-45f7-b528-1c08f3c8a751 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:02.516382 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0215a-0a5a-45f7-b528-1c08f3c8a751 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:02.521023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c801b-972a-4c3e-80b3-266e30fced9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:02.521949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c801b-972a-4c3e-80b3-266e30fced9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.522232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c801b-972a-4c3e-80b3-266e30fced9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.522509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c801b-972a-4c3e-80b3-266e30fced9a admin admin] PATH_INFO: `/v3/projects/004e714b57b844bea3748a8a633e7892/users/cbb768a3cee54f98ab1c2c27ff39cc0d/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.524471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c801b-972a-4c3e-80b3-266e30fced9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=004e714b57b844bea3748a8a633e7892, user_id=cbb768a3cee54f98ab1c2c27ff39cc0d, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:02.527156 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c801b-972a-4c3e-80b3-266e30fced9a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:02.537690 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1288/2572] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:02 2022] PUT /identity/v3/projects/004e714b57b844bea3748a8a633e7892/users/cbb768a3cee54f98ab1c2c27ff39cc0d/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:02.549429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad034b-35de-4c72-85cc-97dc989f58d1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.549729 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad034b-35de-4c72-85cc-97dc989f58d1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.550011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad034b-35de-4c72-85cc-97dc989f58d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.577219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c0ad034b-35de-4c72-85cc-97dc989f58d1 None None] MFA Rules not processed for user `cbb768a3cee54f98ab1c2c27ff39cc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:02.578440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c0ad034b-35de-4c72-85cc-97dc989f58d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:02.598998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:02.652273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1285/2573] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:02.663447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c801b-972a-4c3e-80b3-266e30fced9a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:02.665235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a62c801b-972a-4c3e-80b3-266e30fced9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:02.706244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:02.725744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c801b-972a-4c3e-80b3-266e30fced9a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:02.729901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefd19c-3c98-49de-80e7-c7d80d594f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:02.730555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefd19c-3c98-49de-80e7-c7d80d594f4a service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.730843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefd19c-3c98-49de-80e7-c7d80d594f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.731127 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefd19c-3c98-49de-80e7-c7d80d594f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.740427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefd19c-3c98-49de-80e7-c7d80d594f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:02.741838 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefd19c-3c98-49de-80e7-c7d80d594f4a service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:02.751930 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1289/2574] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:02.790050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad034b-35de-4c72-85cc-97dc989f58d1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:02.791435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c0ad034b-35de-4c72-85cc-97dc989f58d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:02.803724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad034b-35de-4c72-85cc-97dc989f58d1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:02.807295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc04c8-7fb3-42ab-b8b2-6fc61c27cfb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:02.808560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc04c8-7fb3-42ab-b8b2-6fc61c27cfb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:02.808847 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc04c8-7fb3-42ab-b8b2-6fc61c27cfb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:02.809154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc04c8-7fb3-42ab-b8b2-6fc61c27cfb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:02.864611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc04c8-7fb3-42ab-b8b2-6fc61c27cfb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:02.866112 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc04c8-7fb3-42ab-b8b2-6fc61c27cfb9 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:02.875634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1286/2575] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:05.718131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefd19c-3c98-49de-80e7-c7d80d594f4a service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:05.781237 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefd19c-3c98-49de-80e7-c7d80d594f4a service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:05.785966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:05.786830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:05.787115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:05.787395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] PATH_INFO: `/v3/users/381e4a68ab70487ba1ae2511dc867bf2` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:05.788586 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=381e4a68ab70487ba1ae2511dc867bf2)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:05.790541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:05.838581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381e4a68ab70487ba1ae2511dc867bf2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:05.850477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381e4a68ab70487ba1ae2511dc867bf2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:05.858750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381e4a68ab70487ba1ae2511dc867bf2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:05.859490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381e4a68ab70487ba1ae2511dc867bf2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:05.866588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381e4a68ab70487ba1ae2511dc867bf2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:05.873569 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1290/2576] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:05 2022] DELETE /identity/v3/users/381e4a68ab70487ba1ae2511dc867bf2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:05.878179 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfed8e5-13d8-49e9-bc5b-2c7435ab0c0b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:05.878672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfed8e5-13d8-49e9-bc5b-2c7435ab0c0b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:05.878783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfed8e5-13d8-49e9-bc5b-2c7435ab0c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:05.895521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5dfed8e5-13d8-49e9-bc5b-2c7435ab0c0b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:05.896676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5dfed8e5-13d8-49e9-bc5b-2c7435ab0c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:05.956354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1287/2577] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:05.966417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:05.967819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:06.023677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadb696-8008-4973-a48b-ff5d2181eb4a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:06.027920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ced8-5a11-48ef-8a8e-38914790d15d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:06.028563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ced8-5a11-48ef-8a8e-38914790d15d service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:06.028851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ced8-5a11-48ef-8a8e-38914790d15d service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:06.029164 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ced8-5a11-48ef-8a8e-38914790d15d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:06.038285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ced8-5a11-48ef-8a8e-38914790d15d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:06.039715 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ced8-5a11-48ef-8a8e-38914790d15d service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:06.047280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:06.057550 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1291/2578] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:06.068709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfed8e5-13d8-49e9-bc5b-2c7435ab0c0b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:06.070153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5dfed8e5-13d8-49e9-bc5b-2c7435ab0c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:06.082204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfed8e5-13d8-49e9-bc5b-2c7435ab0c0b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:06.086904 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:06.087784 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:06.088073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:06.088386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] PATH_INFO: `/v3/users/fc4c1453eb294b7590e56e7ce3af7800` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:06.090418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc4c1453eb294b7590e56e7ce3af7800)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:06.092520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:06.143597 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc4c1453eb294b7590e56e7ce3af7800'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.148182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:06.158616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc4c1453eb294b7590e56e7ce3af7800'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.166460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc4c1453eb294b7590e56e7ce3af7800'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.175332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc4c1453eb294b7590e56e7ce3af7800'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.176039 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc4c1453eb294b7590e56e7ce3af7800'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.182133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1288/2579] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:06 2022] DELETE /identity/v3/users/fc4c1453eb294b7590e56e7ce3af7800 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:06.186635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e655518-f764-400a-84c7-377550f8ac5d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:06.187045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e655518-f764-400a-84c7-377550f8ac5d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:06.187401 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e655518-f764-400a-84c7-377550f8ac5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:06.207462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7e655518-f764-400a-84c7-377550f8ac5d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:06.208891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7e655518-f764-400a-84c7-377550f8ac5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:06.274794 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1292/2580] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:06.286651 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:06.288098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:06.351602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e655518-f764-400a-84c7-377550f8ac5d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:06.353205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7e655518-f764-400a-84c7-377550f8ac5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:06.355169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dfeca-9ca7-4339-bd2d-96530fdad4fd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:06.363602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07140-1468-4ebf-9ae2-7a2511aeb2cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:06.364279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07140-1468-4ebf-9ae2-7a2511aeb2cb service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:06.364567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07140-1468-4ebf-9ae2-7a2511aeb2cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:06.364846 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07140-1468-4ebf-9ae2-7a2511aeb2cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:06.365985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e655518-f764-400a-84c7-377550f8ac5d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:06.371523 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:06.372466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:06.372826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:06.373200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] PATH_INFO: `/v3/projects/60b17c22c299464cb240eebfb4f4dd5d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:06.374477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b17c22c299464cb240eebfb4f4dd5d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:06.375945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc07140-1468-4ebf-9ae2-7a2511aeb2cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:06.376557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:06.377805 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc07140-1468-4ebf-9ae2-7a2511aeb2cb service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:06.389354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1289/2581] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:06.427976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b17c22c299464cb240eebfb4f4dd5d', 'request_id': 'req-6f4594da-dbbe-4d5b-ae46-d54da75770cc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.435089 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b17c22c299464cb240eebfb4f4dd5d', 'request_id': 'req-6f4594da-dbbe-4d5b-ae46-d54da75770cc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.443030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b17c22c299464cb240eebfb4f4dd5d', 'request_id': 'req-6f4594da-dbbe-4d5b-ae46-d54da75770cc'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.455859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] The token cache is being invalidate because project 60b17c22c299464cb240eebfb4f4dd5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:06.456393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.457066 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.459427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1293/2582] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:06 2022] DELETE /identity/v3/projects/60b17c22c299464cb240eebfb4f4dd5d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:06.463551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07140-1468-4ebf-9ae2-7a2511aeb2cb service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:06.474521 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07140-1468-4ebf-9ae2-7a2511aeb2cb service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:06.547260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:06.548151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:06.548436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:06.548713 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] PATH_INFO: `/v3/users/128232763b3c420c83ae62a3052c4ca9` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:06.550299 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=128232763b3c420c83ae62a3052c4ca9)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:06.551938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:06.604493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128232763b3c420c83ae62a3052c4ca9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.616995 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128232763b3c420c83ae62a3052c4ca9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.625672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128232763b3c420c83ae62a3052c4ca9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.628269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:06.635465 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128232763b3c420c83ae62a3052c4ca9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.636983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128232763b3c420c83ae62a3052c4ca9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.644659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1290/2583] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:06 2022] DELETE /identity/v3/users/128232763b3c420c83ae62a3052c4ca9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:06.673500 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:06.675061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:06.701739 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4594da-dbbe-4d5b-ae46-d54da75770cc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:06.765266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:06.766133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:06.766458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:06.766744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] PATH_INFO: `/v3/projects/14d3650acc0c462eb0aaca8fba7e19aa` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:06.767943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d3650acc0c462eb0aaca8fba7e19aa)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:06.769982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:06.807130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:06.830089 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d3650acc0c462eb0aaca8fba7e19aa', 'request_id': 'req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.836493 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d3650acc0c462eb0aaca8fba7e19aa', 'request_id': 'req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.844122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d3650acc0c462eb0aaca8fba7e19aa', 'request_id': 'req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.855059 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] The token cache is being invalidate because project 14d3650acc0c462eb0aaca8fba7e19aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:06.855346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.855984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:06.857572 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1294/2584] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:06 2022] DELETE /identity/v3/projects/14d3650acc0c462eb0aaca8fba7e19aa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:06.862133 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:06.863568 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:06.877210 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadde080-47d4-42b8-b256-d0dd370aded1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:06.929294 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:06.930869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:06.938021 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:06.939401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:06.939762 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:06.940170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] PATH_INFO: `/v3/users/83f4edfb164248fa9515a894854f3602` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:06.941550 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f4edfb164248fa9515a894854f3602)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:06.943965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:06.997362 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f4edfb164248fa9515a894854f3602'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.006413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5c3d1-485c-42cd-9c59-98fb3fa6de5b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.009560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f4edfb164248fa9515a894854f3602'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.011378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.012243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.012566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.012851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] PATH_INFO: `/v3/projects/f7df03587396443a902f48c65da5a010` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.014128 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7df03587396443a902f48c65da5a010)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.016520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.017387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f4edfb164248fa9515a894854f3602'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.024775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f4edfb164248fa9515a894854f3602'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.025454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f4edfb164248fa9515a894854f3602'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.031624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1291/2585] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:06 2022] DELETE /identity/v3/users/83f4edfb164248fa9515a894854f3602 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:07.065022 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7df03587396443a902f48c65da5a010', 'request_id': 'req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.072314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7df03587396443a902f48c65da5a010', 'request_id': 'req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.082444 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7df03587396443a902f48c65da5a010', 'request_id': 'req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.096063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] The token cache is being invalidate because project f7df03587396443a902f48c65da5a010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:07.096362 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.097039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.098716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1295/2586] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:06 2022] DELETE /identity/v3/projects/f7df03587396443a902f48c65da5a010 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:07.104468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.106969 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:07.188400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc079db7-ef91-41e4-b892-f24272998258 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.202115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:07.255147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.256068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.256388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.256672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] PATH_INFO: `/v3/users/cbb768a3cee54f98ab1c2c27ff39cc0d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.257959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb768a3cee54f98ab1c2c27ff39cc0d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.260287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.312545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb768a3cee54f98ab1c2c27ff39cc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.331871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb768a3cee54f98ab1c2c27ff39cc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.339659 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb768a3cee54f98ab1c2c27ff39cc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.343881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.345394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:07.348113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb768a3cee54f98ab1c2c27ff39cc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.348801 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e70081fd-50e2-44d0-b6bb-acca24db11bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb768a3cee54f98ab1c2c27ff39cc0d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.357132 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1292/2587] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:07 2022] DELETE /identity/v3/users/cbb768a3cee54f98ab1c2c27ff39cc0d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:07.364706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3daa6d-c24a-41b5-afd8-35cb603c7c1c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.412990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:07.431631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.432527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.432952 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.433297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] PATH_INFO: `/v3/projects/7b125b09df1f4c12bc773103b95f6e08` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.434524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b125b09df1f4c12bc773103b95f6e08)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.436563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.485693 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b125b09df1f4c12bc773103b95f6e08', 'request_id': 'req-33a12b20-7e51-4f22-b067-4e8edfae9ba4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.492075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b125b09df1f4c12bc773103b95f6e08', 'request_id': 'req-33a12b20-7e51-4f22-b067-4e8edfae9ba4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.501741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b125b09df1f4c12bc773103b95f6e08', 'request_id': 'req-33a12b20-7e51-4f22-b067-4e8edfae9ba4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.513887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] The token cache is being invalidate because project 7b125b09df1f4c12bc773103b95f6e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:07.514181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.514907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-33a12b20-7e51-4f22-b067-4e8edfae9ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.516482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1296/2588] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:07 2022] DELETE /identity/v3/projects/7b125b09df1f4c12bc773103b95f6e08 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:07.521401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ae2b-7746-4677-9f90-579c6d5a1229 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.521680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ae2b-7746-4677-9f90-579c6d5a1229 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.521965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ae2b-7746-4677-9f90-579c6d5a1229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.541328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ce75ae2b-7746-4677-9f90-579c6d5a1229 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:07.543204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ce75ae2b-7746-4677-9f90-579c6d5a1229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:07.605979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1293/2589] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:07.613615 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e13c-691d-437a-b30e-225ef2a49fd5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.613917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e13c-691d-437a-b30e-225ef2a49fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.614206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e13c-691d-437a-b30e-225ef2a49fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.634983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b015e13c-691d-437a-b30e-225ef2a49fd5 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:07.636352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b015e13c-691d-437a-b30e-225ef2a49fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:07.644847 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1297/2590] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:07.649078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75ae2b-7746-4677-9f90-579c6d5a1229 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.659860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015e13c-691d-437a-b30e-225ef2a49fd5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.661744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b015e13c-691d-437a-b30e-225ef2a49fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:07.663881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75ae2b-7746-4677-9f90-579c6d5a1229 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.670118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6699e4e-65d4-4775-a7b0-46f709f96e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.671054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6699e4e-65d4-4775-a7b0-46f709f96e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.671398 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6699e4e-65d4-4775-a7b0-46f709f96e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.671652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6699e4e-65d4-4775-a7b0-46f709f96e53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.672436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6699e4e-65d4-4775-a7b0-46f709f96e53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:15:07.672704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6699e4e-65d4-4775-a7b0-46f709f96e53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.674374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6699e4e-65d4-4775-a7b0-46f709f96e53 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.675267 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015e13c-691d-437a-b30e-225ef2a49fd5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.680011 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.680994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.681484 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.681858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] PATH_INFO: `/v3/projects/004e714b57b844bea3748a8a633e7892` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.683167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] RBAC: Authorizing `identity:delete_project(project_id=004e714b57b844bea3748a8a633e7892)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.685154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.687190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1294/2591] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:15:07.691364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6699e4e-65d4-4775-a7b0-46f709f96e53 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.703293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6699e4e-65d4-4775-a7b0-46f709f96e53 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.709017 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b009-7b23-4935-b1a0-d62366122a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.709654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b009-7b23-4935-b1a0-d62366122a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.709943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b009-7b23-4935-b1a0-d62366122a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.710229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b009-7b23-4935-b1a0-d62366122a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.710965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98b009-7b23-4935-b1a0-d62366122a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.712900 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98b009-7b23-4935-b1a0-d62366122a84 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.745384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1295/2592] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:07 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:15:07.748938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004e714b57b844bea3748a8a633e7892', 'request_id': 'req-27698f16-1c61-46cc-9493-0c51dd70f04c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.749436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b009-7b23-4935-b1a0-d62366122a84 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.755189 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:07.756594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004e714b57b844bea3748a8a633e7892', 'request_id': 'req-27698f16-1c61-46cc-9493-0c51dd70f04c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.765958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004e714b57b844bea3748a8a633e7892', 'request_id': 'req-27698f16-1c61-46cc-9493-0c51dd70f04c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.778791 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] The token cache is being invalidate because project 004e714b57b844bea3748a8a633e7892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:07.779085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.779747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-27698f16-1c61-46cc-9493-0c51dd70f04c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:07.781407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1298/2593] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:07 2022] DELETE /identity/v3/projects/004e714b57b844bea3748a8a633e7892 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:07.787474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4bc39-b127-46e1-b4b3-6e6d44232db4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.787780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4bc39-b127-46e1-b4b3-6e6d44232db4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.788063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4bc39-b127-46e1-b4b3-6e6d44232db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.806949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a1c4bc39-b127-46e1-b4b3-6e6d44232db4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:07.808200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a1c4bc39-b127-46e1-b4b3-6e6d44232db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:07.822859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b009-7b23-4935-b1a0-d62366122a84 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.828440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a4ccd-6e32-4306-8bd1-a244b0ffa77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.829239 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a4ccd-6e32-4306-8bd1-a244b0ffa77b admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.829425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a4ccd-6e32-4306-8bd1-a244b0ffa77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.829709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a4ccd-6e32-4306-8bd1-a244b0ffa77b admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.830430 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a4ccd-6e32-4306-8bd1-a244b0ffa77b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.832402 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a4ccd-6e32-4306-8bd1-a244b0ffa77b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.853277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1296/2594] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:07 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:07.855355 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:07.857681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a4ccd-6e32-4306-8bd1-a244b0ffa77b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.859631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4b5a4ccd-6e32-4306-8bd1-a244b0ffa77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:07.873026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a4ccd-6e32-4306-8bd1-a244b0ffa77b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.873271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1299/2595] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:07.881596 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d5a3c-40ee-4c5f-8691-fcb246ce19f4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.881887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d5a3c-40ee-4c5f-8691-fcb246ce19f4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.882167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d5a3c-40ee-4c5f-8691-fcb246ce19f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.882442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df7eb3-56f9-4926-ac5f-1a6203ec7226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.883261 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df7eb3-56f9-4926-ac5f-1a6203ec7226 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.883693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df7eb3-56f9-4926-ac5f-1a6203ec7226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.884102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df7eb3-56f9-4926-ac5f-1a6203ec7226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.884944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df7eb3-56f9-4926-ac5f-1a6203ec7226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.887453 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df7eb3-56f9-4926-ac5f-1a6203ec7226 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.902059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1297/2596] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:15:07.902278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-544d5a3c-40ee-4c5f-8691-fcb246ce19f4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:07.903107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-544d5a3c-40ee-4c5f-8691-fcb246ce19f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:07.903310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df7eb3-56f9-4926-ac5f-1a6203ec7226 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.911107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1300/2597] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:07.913042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df7eb3-56f9-4926-ac5f-1a6203ec7226 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.915191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d5a3c-40ee-4c5f-8691-fcb246ce19f4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.917990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca1216-8204-4113-892e-88776acfdeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.918642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca1216-8204-4113-892e-88776acfdeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.918924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca1216-8204-4113-892e-88776acfdeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.919198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca1216-8204-4113-892e-88776acfdeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.919915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca1216-8204-4113-892e-88776acfdeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.921541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca1216-8204-4113-892e-88776acfdeb3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.924891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d5a3c-40ee-4c5f-8691-fcb246ce19f4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.930552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243b819-aae7-41e0-8ded-dd7683191397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.931197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243b819-aae7-41e0-8ded-dd7683191397 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.931515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243b819-aae7-41e0-8ded-dd7683191397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.931634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1298/2598] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:07.931840 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243b819-aae7-41e0-8ded-dd7683191397 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.932672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243b819-aae7-41e0-8ded-dd7683191397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:15:07.932984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243b819-aae7-41e0-8ded-dd7683191397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.934724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243b819-aae7-41e0-8ded-dd7683191397 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.935409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca1216-8204-4113-892e-88776acfdeb3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.945485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca1216-8204-4113-892e-88776acfdeb3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.951478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1301/2599] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:15:07.951631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7873b69-5851-478a-a0fb-61077142ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.951785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7873b69-5851-478a-a0fb-61077142ee70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.952005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7873b69-5851-478a-a0fb-61077142ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.952287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7873b69-5851-478a-a0fb-61077142ee70 admin admin] PATH_INFO: `/v3/projects/8551c5bb991149b3a85cdf23af581efa/users/e9df4b165c784bac914b0e91dfbe6e39/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.952457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243b819-aae7-41e0-8ded-dd7683191397 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:07.962203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243b819-aae7-41e0-8ded-dd7683191397 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:07.967538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964e18c-c962-420b-a96d-6b5866d1eb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:07.968173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964e18c-c962-420b-a96d-6b5866d1eb8f admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.968279 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7873b69-5851-478a-a0fb-61077142ee70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8551c5bb991149b3a85cdf23af581efa, user_id=e9df4b165c784bac914b0e91dfbe6e39, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.968459 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964e18c-c962-420b-a96d-6b5866d1eb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.968737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964e18c-c962-420b-a96d-6b5866d1eb8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.969450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964e18c-c962-420b-a96d-6b5866d1eb8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:07.970901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7873b69-5851-478a-a0fb-61077142ee70 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.971342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964e18c-c962-420b-a96d-6b5866d1eb8f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:07.984176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1299/2600] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:07 2022] PUT /identity/v3/projects/8551c5bb991149b3a85cdf23af581efa/users/e9df4b165c784bac914b0e91dfbe6e39/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:07.994596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba3988-910b-4e97-948b-fb8ea968b2d7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:07.994902 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba3988-910b-4e97-948b-fb8ea968b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:07.995177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba3988-910b-4e97-948b-fb8ea968b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:07.997949 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1302/2601] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:07 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:08.001846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964e18c-c962-420b-a96d-6b5866d1eb8f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.026079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-75ba3988-910b-4e97-948b-fb8ea968b2d7 None None] MFA Rules not processed for user `e9df4b165c784bac914b0e91dfbe6e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:08.028323 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-75ba3988-910b-4e97-948b-fb8ea968b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:08.064651 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964e18c-c962-420b-a96d-6b5866d1eb8f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.069658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572924-d99a-4b2d-9a8b-367f4433ea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.070321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572924-d99a-4b2d-9a8b-367f4433ea7c admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.070612 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572924-d99a-4b2d-9a8b-367f4433ea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.070971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572924-d99a-4b2d-9a8b-367f4433ea7c admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.071667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572924-d99a-4b2d-9a8b-367f4433ea7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.074238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572924-d99a-4b2d-9a8b-367f4433ea7c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.094268 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1303/2602] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:08 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:08.096519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1300/2603] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:08.098196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572924-d99a-4b2d-9a8b-367f4433ea7c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.100639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba3988-910b-4e97-948b-fb8ea968b2d7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.108146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572924-d99a-4b2d-9a8b-367f4433ea7c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.110903 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba3988-910b-4e97-948b-fb8ea968b2d7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.112793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f87d9-067f-45d4-a7c0-2bfb159bc2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.113476 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f87d9-067f-45d4-a7c0-2bfb159bc2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.113765 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f87d9-067f-45d4-a7c0-2bfb159bc2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.114051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f87d9-067f-45d4-a7c0-2bfb159bc2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.114774 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f87d9-067f-45d4-a7c0-2bfb159bc2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.116099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659859fd-ee3e-4cd6-abf5-d8f9d6964b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.116359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f87d9-067f-45d4-a7c0-2bfb159bc2b3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.116839 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659859fd-ee3e-4cd6-abf5-d8f9d6964b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.117154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659859fd-ee3e-4cd6-abf5-d8f9d6964b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.117441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659859fd-ee3e-4cd6-abf5-d8f9d6964b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.118134 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659859fd-ee3e-4cd6-abf5-d8f9d6964b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.120567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659859fd-ee3e-4cd6-abf5-d8f9d6964b84 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.127280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1304/2604] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:15:08.131503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f87d9-067f-45d4-a7c0-2bfb159bc2b3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.142191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f87d9-067f-45d4-a7c0-2bfb159bc2b3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.146925 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e0184-9012-44c5-999d-bf50383326bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.147101 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1301/2605] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:08 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:08.149904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0184-9012-44c5-999d-bf50383326bc admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.150189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0184-9012-44c5-999d-bf50383326bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.150468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0184-9012-44c5-999d-bf50383326bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.150780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659859fd-ee3e-4cd6-abf5-d8f9d6964b84 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.151223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e0184-9012-44c5-999d-bf50383326bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.152808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e0184-9012-44c5-999d-bf50383326bc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.163851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:08.164056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1305/2606] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:08.167751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e0184-9012-44c5-999d-bf50383326bc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.178069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e0184-9012-44c5-999d-bf50383326bc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.217184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659859fd-ee3e-4cd6-abf5-d8f9d6964b84 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.222627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41685c-0bfe-4a1c-98bb-bc2b7183901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.223314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41685c-0bfe-4a1c-98bb-bc2b7183901f admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.223603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41685c-0bfe-4a1c-98bb-bc2b7183901f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.223895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41685c-0bfe-4a1c-98bb-bc2b7183901f admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.224647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41685c-0bfe-4a1c-98bb-bc2b7183901f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.226601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41685c-0bfe-4a1c-98bb-bc2b7183901f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.241313 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd88782-f164-486c-a536-dd29e6af6a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.242203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd88782-f164-486c-a536-dd29e6af6a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.242485 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd88782-f164-486c-a536-dd29e6af6a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.242761 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd88782-f164-486c-a536-dd29e6af6a63 admin admin] PATH_INFO: `/v3/projects/e31c137d747543a0b5147202dce999f6/users/26e04bbbbd614177bcc016fc05bfec4b/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.251530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1302/2607] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:08 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:15:08.252812 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41685c-0bfe-4a1c-98bb-bc2b7183901f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.258304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd88782-f164-486c-a536-dd29e6af6a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31c137d747543a0b5147202dce999f6, user_id=26e04bbbbd614177bcc016fc05bfec4b, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.261656 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd88782-f164-486c-a536-dd29e6af6a63 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.263098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41685c-0bfe-4a1c-98bb-bc2b7183901f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.268295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa67556-b3ae-4b17-a77a-a519ed40acb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.269024 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa67556-b3ae-4b17-a77a-a519ed40acb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.269382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa67556-b3ae-4b17-a77a-a519ed40acb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.269598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa67556-b3ae-4b17-a77a-a519ed40acb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.270325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa67556-b3ae-4b17-a77a-a519ed40acb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.271710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:08.271974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa67556-b3ae-4b17-a77a-a519ed40acb8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.277133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1306/2608] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:08 2022] PUT /identity/v3/projects/e31c137d747543a0b5147202dce999f6/users/26e04bbbbd614177bcc016fc05bfec4b/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:08.283293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1303/2609] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:15:08.288992 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8772-5f94-496b-98c9-955a999c4bf2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.289220 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa67556-b3ae-4b17-a77a-a519ed40acb8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.289374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8772-5f94-496b-98c9-955a999c4bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.289649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8772-5f94-496b-98c9-955a999c4bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.299993 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa67556-b3ae-4b17-a77a-a519ed40acb8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.315610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8e4a8772-5f94-496b-98c9-955a999c4bf2 None None] MFA Rules not processed for user `26e04bbbbd614177bcc016fc05bfec4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:08.316815 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8e4a8772-5f94-496b-98c9-955a999c4bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:08.366760 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fcefd-c568-4f0b-9b4e-1942fe4691a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.367646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fcefd-c568-4f0b-9b4e-1942fe4691a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.367930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fcefd-c568-4f0b-9b4e-1942fe4691a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.368208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fcefd-c568-4f0b-9b4e-1942fe4691a5 admin admin] PATH_INFO: `/v3/projects/bc9bfc9354a146af9e4ddc3cee8957e6/users/54cf0f55aa6945039b281862ba7c37f1/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.383695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fcefd-c568-4f0b-9b4e-1942fe4691a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9bfc9354a146af9e4ddc3cee8957e6, user_id=54cf0f55aa6945039b281862ba7c37f1, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.387203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fcefd-c568-4f0b-9b4e-1942fe4691a5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.387565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1307/2610] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:08.391688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a8772-5f94-496b-98c9-955a999c4bf2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.403133 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a8772-5f94-496b-98c9-955a999c4bf2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.403742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1304/2611] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:08 2022] PUT /identity/v3/projects/bc9bfc9354a146af9e4ddc3cee8957e6/users/54cf0f55aa6945039b281862ba7c37f1/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:08.407745 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fcefd-c568-4f0b-9b4e-1942fe4691a5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.462567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5a255-2323-4a9b-88b8-ef568fd3ae4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.463502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5a255-2323-4a9b-88b8-ef568fd3ae4e admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.463782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5a255-2323-4a9b-88b8-ef568fd3ae4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.464058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5a255-2323-4a9b-88b8-ef568fd3ae4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.464745 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5a255-2323-4a9b-88b8-ef568fd3ae4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.466668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5a255-2323-4a9b-88b8-ef568fd3ae4e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.471881 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fcefd-c568-4f0b-9b4e-1942fe4691a5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.477545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201aba9-d3e7-4983-bc81-4f1b6bbdff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.478214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201aba9-d3e7-4983-bc81-4f1b6bbdff11 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.478537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201aba9-d3e7-4983-bc81-4f1b6bbdff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.478820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201aba9-d3e7-4983-bc81-4f1b6bbdff11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.480092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201aba9-d3e7-4983-bc81-4f1b6bbdff11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.481742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201aba9-d3e7-4983-bc81-4f1b6bbdff11 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.492779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1305/2612] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:15:08.493045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1308/2613] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:08 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:15:08.497424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201aba9-d3e7-4983-bc81-4f1b6bbdff11 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.497561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5a255-2323-4a9b-88b8-ef568fd3ae4e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.507373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201aba9-d3e7-4983-bc81-4f1b6bbdff11 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.515160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:08.561341 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5a255-2323-4a9b-88b8-ef568fd3ae4e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.565738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7973c-280c-4539-a566-8399e55f7e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.566104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c2f33-cc01-4dec-8aff-150acd0df17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.566629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7973c-280c-4539-a566-8399e55f7e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.566766 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c2f33-cc01-4dec-8aff-150acd0df17e admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.566921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7973c-280c-4539-a566-8399e55f7e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.567052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c2f33-cc01-4dec-8aff-150acd0df17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.567205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7973c-280c-4539-a566-8399e55f7e89 admin admin] PATH_INFO: `/v3/projects/bc9bfc9354a146af9e4ddc3cee8957e6/users/54cf0f55aa6945039b281862ba7c37f1/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.567333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c2f33-cc01-4dec-8aff-150acd0df17e admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.568023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c2f33-cc01-4dec-8aff-150acd0df17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.569536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7973c-280c-4539-a566-8399e55f7e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9bfc9354a146af9e4ddc3cee8957e6, user_id=54cf0f55aa6945039b281862ba7c37f1, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.570333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c2f33-cc01-4dec-8aff-150acd0df17e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.572170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7973c-280c-4539-a566-8399e55f7e89 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.584303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1306/2614] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:08 2022] PUT /identity/v3/projects/bc9bfc9354a146af9e4ddc3cee8957e6/users/54cf0f55aa6945039b281862ba7c37f1/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:08.592212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1309/2615] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:08 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:08.594172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7973c-280c-4539-a566-8399e55f7e89 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.595179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14108930-3f53-451d-8ca7-043be14ee002 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.595487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14108930-3f53-451d-8ca7-043be14ee002 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.595795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14108930-3f53-451d-8ca7-043be14ee002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.602389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:08.624210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-14108930-3f53-451d-8ca7-043be14ee002 None None] MFA Rules not processed for user `54cf0f55aa6945039b281862ba7c37f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:08.625517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-14108930-3f53-451d-8ca7-043be14ee002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:08.662843 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7973c-280c-4539-a566-8399e55f7e89 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.668217 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93968dc-dc5f-4788-8fd3-1f957c508255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.668882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93968dc-dc5f-4788-8fd3-1f957c508255 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.669196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93968dc-dc5f-4788-8fd3-1f957c508255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.669508 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93968dc-dc5f-4788-8fd3-1f957c508255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.670238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93968dc-dc5f-4788-8fd3-1f957c508255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.671859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93968dc-dc5f-4788-8fd3-1f957c508255 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.682381 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1307/2616] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:08.686963 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93968dc-dc5f-4788-8fd3-1f957c508255 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.696835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93968dc-dc5f-4788-8fd3-1f957c508255 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.698262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1310/2617] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:08 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:08.702126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a7631-824f-482c-ac41-2c8c88f435b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.703008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a7631-824f-482c-ac41-2c8c88f435b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.703297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a7631-824f-482c-ac41-2c8c88f435b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.703578 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a7631-824f-482c-ac41-2c8c88f435b3 admin admin] PATH_INFO: `/v3/projects/23724f7f736446ada90fb3cd53048390/users/d50c27e543734664901483f8a29b8646/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.709524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14108930-3f53-451d-8ca7-043be14ee002 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.711056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-14108930-3f53-451d-8ca7-043be14ee002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:08.718827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a7631-824f-482c-ac41-2c8c88f435b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23724f7f736446ada90fb3cd53048390, user_id=d50c27e543734664901483f8a29b8646, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.721530 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a7631-824f-482c-ac41-2c8c88f435b3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.735099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1308/2618] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:08 2022] PUT /identity/v3/projects/23724f7f736446ada90fb3cd53048390/users/d50c27e543734664901483f8a29b8646/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:08.738833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a7631-824f-482c-ac41-2c8c88f435b3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.770616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14108930-3f53-451d-8ca7-043be14ee002 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.803687 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a7631-824f-482c-ac41-2c8c88f435b3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.809078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64dea-eef3-43c2-94bf-d5ec64c639d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.809783 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64dea-eef3-43c2-94bf-d5ec64c639d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.810052 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64dea-eef3-43c2-94bf-d5ec64c639d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.810335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64dea-eef3-43c2-94bf-d5ec64c639d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.811067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c64dea-eef3-43c2-94bf-d5ec64c639d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.812668 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c64dea-eef3-43c2-94bf-d5ec64c639d2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.820605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f45571-b5f1-4094-a893-5a1c7725171c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.821285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f45571-b5f1-4094-a893-5a1c7725171c service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.821574 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f45571-b5f1-4094-a893-5a1c7725171c service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.821852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f45571-b5f1-4094-a893-5a1c7725171c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.822849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1309/2619] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:15:08.826992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64dea-eef3-43c2-94bf-d5ec64c639d2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.837124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64dea-eef3-43c2-94bf-d5ec64c639d2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.842043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22818a86-a03d-4e60-88aa-29e0d54a0aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:08.843345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22818a86-a03d-4e60-88aa-29e0d54a0aab admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.843634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22818a86-a03d-4e60-88aa-29e0d54a0aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.844326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22818a86-a03d-4e60-88aa-29e0d54a0aab admin admin] PATH_INFO: `/v3/projects/23724f7f736446ada90fb3cd53048390/users/d50c27e543734664901483f8a29b8646/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.846596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22818a86-a03d-4e60-88aa-29e0d54a0aab admin admin] RBAC: Authorizing `identity:create_grant(project_id=23724f7f736446ada90fb3cd53048390, user_id=d50c27e543734664901483f8a29b8646, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.849332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22818a86-a03d-4e60-88aa-29e0d54a0aab admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.856917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:08.862316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1310/2620] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:08 2022] PUT /identity/v3/projects/23724f7f736446ada90fb3cd53048390/users/d50c27e543734664901483f8a29b8646/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:08.873640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4897a7-efb8-4e01-845b-a1d8fbd17210 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:08.873971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4897a7-efb8-4e01-845b-a1d8fbd17210 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:08.874199 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4897a7-efb8-4e01-845b-a1d8fbd17210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:08.886799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:08.889249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f45571-b5f1-4094-a893-5a1c7725171c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:08.890881 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f45571-b5f1-4094-a893-5a1c7725171c service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:08.902441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1311/2621] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:15:08.903627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1c4897a7-efb8-4e01-845b-a1d8fbd17210 None None] MFA Rules not processed for user `d50c27e543734664901483f8a29b8646`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:08.904894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1c4897a7-efb8-4e01-845b-a1d8fbd17210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:08.946350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f45571-b5f1-4094-a893-5a1c7725171c service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:08.947787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-68f45571-b5f1-4094-a893-5a1c7725171c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:08.959089 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f45571-b5f1-4094-a893-5a1c7725171c service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:08.984251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1311/2622] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:08.995406 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4897a7-efb8-4e01-845b-a1d8fbd17210 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:09.010929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8dca5d-924c-4b71-a503-73aacf089e7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:09.011573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dca5d-924c-4b71-a503-73aacf089e7c service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:09.011858 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dca5d-924c-4b71-a503-73aacf089e7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:09.012162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dca5d-924c-4b71-a503-73aacf089e7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:09.053291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4897a7-efb8-4e01-845b-a1d8fbd17210 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:09.057028 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c5f2-43c4-429d-9894-f70fda0ace77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:09.057871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c5f2-43c4-429d-9894-f70fda0ace77 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:09.058155 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c5f2-43c4-429d-9894-f70fda0ace77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:09.058576 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c5f2-43c4-429d-9894-f70fda0ace77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:09.067877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01c5f2-43c4-429d-9894-f70fda0ace77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:09.069652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01c5f2-43c4-429d-9894-f70fda0ace77 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:09.069878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dca5d-924c-4b71-a503-73aacf089e7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:09.071448 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dca5d-924c-4b71-a503-73aacf089e7c service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:09.079260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1312/2623] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:15:09.080623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1312/2624] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:08 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:15:09.625946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c5f2-43c4-429d-9894-f70fda0ace77 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:09.687816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c5f2-43c4-429d-9894-f70fda0ace77 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:09.692325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:09.693222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:09.693773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:09.694055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] PATH_INFO: `/v3/users/26e04bbbbd614177bcc016fc05bfec4b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:09.695242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e04bbbbd614177bcc016fc05bfec4b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:09.697207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:09.731975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:09.754591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e04bbbbd614177bcc016fc05bfec4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:09.766567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e04bbbbd614177bcc016fc05bfec4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:09.774007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e04bbbbd614177bcc016fc05bfec4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:09.781922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e04bbbbd614177bcc016fc05bfec4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:09.782505 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e04bbbbd614177bcc016fc05bfec4b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:09.789049 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1313/2625] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:09 2022] DELETE /identity/v3/users/26e04bbbbd614177bcc016fc05bfec4b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:09.794177 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa0d9c-c312-4c37-b6e4-46f7db0cfcf5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:09.794677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa0d9c-c312-4c37-b6e4-46f7db0cfcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:09.794899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa0d9c-c312-4c37-b6e4-46f7db0cfcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:09.812290 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-80fa0d9c-c312-4c37-b6e4-46f7db0cfcf5 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:09.813521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-80fa0d9c-c312-4c37-b6e4-46f7db0cfcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:09.871673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1313/2626] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:09.881309 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:09.882764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:09.946948 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ca870-45e9-403e-b177-7e3e823114e3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:09.950673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773a584-9440-4753-9561-1bb36f97fff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:09.951325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773a584-9440-4753-9561-1bb36f97fff1 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:09.951611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773a584-9440-4753-9561-1bb36f97fff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:09.951891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773a584-9440-4753-9561-1bb36f97fff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:09.961012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773a584-9440-4753-9561-1bb36f97fff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:09.962476 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773a584-9440-4753-9561-1bb36f97fff1 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:09.972540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1314/2627] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:10.251614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa0d9c-c312-4c37-b6e4-46f7db0cfcf5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:10.253146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-80fa0d9c-c312-4c37-b6e4-46f7db0cfcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:10.268106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa0d9c-c312-4c37-b6e4-46f7db0cfcf5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:10.277113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:10.277113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:10.277113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:10.277113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] PATH_INFO: `/v3/projects/e31c137d747543a0b5147202dce999f6` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:10.278009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e31c137d747543a0b5147202dce999f6)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:10.278371 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:10.324110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:10.327121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e31c137d747543a0b5147202dce999f6', 'request_id': 'req-7b9b3836-3d0b-4649-814f-f58d6d11bc64'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.334491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e31c137d747543a0b5147202dce999f6', 'request_id': 'req-7b9b3836-3d0b-4649-814f-f58d6d11bc64'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.342113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e31c137d747543a0b5147202dce999f6', 'request_id': 'req-7b9b3836-3d0b-4649-814f-f58d6d11bc64'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.352787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] The token cache is being invalidate because project e31c137d747543a0b5147202dce999f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:10.353076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.353749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.355304 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1314/2628] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:10 2022] DELETE /identity/v3/projects/e31c137d747543a0b5147202dce999f6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:10.359418 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773a584-9440-4753-9561-1bb36f97fff1 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:10.368977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773a584-9440-4753-9561-1bb36f97fff1 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:10.427198 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:10.428065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:10.428351 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:10.428627 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] PATH_INFO: `/v3/users/d50c27e543734664901483f8a29b8646` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:10.429992 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d50c27e543734664901483f8a29b8646)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:10.431907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:10.482004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd50c27e543734664901483f8a29b8646'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.487067 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:10.498526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd50c27e543734664901483f8a29b8646'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.506149 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd50c27e543734664901483f8a29b8646'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.514538 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd50c27e543734664901483f8a29b8646'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.515392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85351706-9a49-4661-b64e-06c4fcd6ca5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd50c27e543734664901483f8a29b8646'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.522123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1315/2629] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:10 2022] DELETE /identity/v3/users/d50c27e543734664901483f8a29b8646 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:10.812419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:10.815533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:10.832210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b3836-3d0b-4649-814f-f58d6d11bc64 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:10.919005 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:10.919854 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:10.920132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:10.920405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] PATH_INFO: `/v3/projects/23724f7f736446ada90fb3cd53048390` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:10.921598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23724f7f736446ada90fb3cd53048390)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:10.923490 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:10.969428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23724f7f736446ada90fb3cd53048390', 'request_id': 'req-9c95555a-2a30-48ea-b56f-d79e0f671ae2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.975668 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23724f7f736446ada90fb3cd53048390', 'request_id': 'req-9c95555a-2a30-48ea-b56f-d79e0f671ae2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.983676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23724f7f736446ada90fb3cd53048390', 'request_id': 'req-9c95555a-2a30-48ea-b56f-d79e0f671ae2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.994214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] The token cache is being invalidate because project 23724f7f736446ada90fb3cd53048390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:10.994503 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.995688 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9c95555a-2a30-48ea-b56f-d79e0f671ae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:10.997266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1315/2630] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:10 2022] DELETE /identity/v3/projects/23724f7f736446ada90fb3cd53048390 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:11.002312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3559a8e-f00b-4b55-bc12-9a0d6bfc59e8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.002604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3559a8e-f00b-4b55-bc12-9a0d6bfc59e8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.002888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3559a8e-f00b-4b55-bc12-9a0d6bfc59e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.020994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e3559a8e-f00b-4b55-bc12-9a0d6bfc59e8 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:11.022241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e3559a8e-f00b-4b55-bc12-9a0d6bfc59e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:11.081536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1316/2631] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:11.089412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ea1b-733e-4408-b633-cddbf76aa7b4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.089827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ea1b-733e-4408-b633-cddbf76aa7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.090114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ea1b-733e-4408-b633-cddbf76aa7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.109342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4193ea1b-733e-4408-b633-cddbf76aa7b4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:11.110911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4193ea1b-733e-4408-b633-cddbf76aa7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:11.118936 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1316/2632] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:11.123794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3559a8e-f00b-4b55-bc12-9a0d6bfc59e8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:11.133512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3559a8e-f00b-4b55-bc12-9a0d6bfc59e8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:11.138117 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b155b2-d918-4fc0-a187-7f6bef2e829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:11.138768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b155b2-d918-4fc0-a187-7f6bef2e829a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.139055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b155b2-d918-4fc0-a187-7f6bef2e829a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.139331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b155b2-d918-4fc0-a187-7f6bef2e829a admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.140056 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b155b2-d918-4fc0-a187-7f6bef2e829a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:15:11.140363 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b155b2-d918-4fc0-a187-7f6bef2e829a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:11.141996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b155b2-d918-4fc0-a187-7f6bef2e829a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:11.154194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1317/2633] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:15:11.158590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193ea1b-733e-4408-b633-cddbf76aa7b4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:11.168421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193ea1b-733e-4408-b633-cddbf76aa7b4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:11.173013 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8e13b-1d2e-4982-ace8-aba546ed3b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:11.173680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8e13b-1d2e-4982-ace8-aba546ed3b2d admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.173960 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8e13b-1d2e-4982-ace8-aba546ed3b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.174234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8e13b-1d2e-4982-ace8-aba546ed3b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.174938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8e13b-1d2e-4982-ace8-aba546ed3b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:11.176810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8e13b-1d2e-4982-ace8-aba546ed3b2d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:11.204280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1317/2634] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:11 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:11.208515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b155b2-d918-4fc0-a187-7f6bef2e829a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:11.219306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b155b2-d918-4fc0-a187-7f6bef2e829a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:11.277023 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b965172-7596-4874-aead-f1158247440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:11.277725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b965172-7596-4874-aead-f1158247440f admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.278012 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b965172-7596-4874-aead-f1158247440f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.278288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b965172-7596-4874-aead-f1158247440f admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.278983 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b965172-7596-4874-aead-f1158247440f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:11.280914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b965172-7596-4874-aead-f1158247440f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:11.300720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1318/2635] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:11 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:11.304883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8e13b-1d2e-4982-ace8-aba546ed3b2d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:11.315872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8e13b-1d2e-4982-ace8-aba546ed3b2d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:11.320758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f992403-9332-4b0a-9392-c4098cd33c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:11.321447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f992403-9332-4b0a-9392-c4098cd33c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.321732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f992403-9332-4b0a-9392-c4098cd33c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.322007 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f992403-9332-4b0a-9392-c4098cd33c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.322748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f992403-9332-4b0a-9392-c4098cd33c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:11.324313 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f992403-9332-4b0a-9392-c4098cd33c25 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:11.334981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1318/2636] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:11.339607 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b965172-7596-4874-aead-f1158247440f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:11.349635 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b965172-7596-4874-aead-f1158247440f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:11.354320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87177c50-24dd-4356-8722-6bc388d00afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:11.355504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87177c50-24dd-4356-8722-6bc388d00afc admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.355826 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87177c50-24dd-4356-8722-6bc388d00afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.356110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87177c50-24dd-4356-8722-6bc388d00afc admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.356838 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87177c50-24dd-4356-8722-6bc388d00afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:11.358489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87177c50-24dd-4356-8722-6bc388d00afc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:11.370547 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1319/2637] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:11.374584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f992403-9332-4b0a-9392-c4098cd33c25 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:11.384649 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f992403-9332-4b0a-9392-c4098cd33c25 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:11.389757 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b90a60-5e51-4ca4-84e9-b8033c1e1f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:11.390643 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b90a60-5e51-4ca4-84e9-b8033c1e1f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.390920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b90a60-5e51-4ca4-84e9-b8033c1e1f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.391194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b90a60-5e51-4ca4-84e9-b8033c1e1f1f admin admin] PATH_INFO: `/v3/projects/b98858235df741bab2a3fb91a470df63/users/d672ced33c484f00a08630740010d889/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.406558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b90a60-5e51-4ca4-84e9-b8033c1e1f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98858235df741bab2a3fb91a470df63, user_id=d672ced33c484f00a08630740010d889, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:11.409337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b90a60-5e51-4ca4-84e9-b8033c1e1f1f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:11.419905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:11.425506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1319/2638] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:11 2022] PUT /identity/v3/projects/b98858235df741bab2a3fb91a470df63/users/d672ced33c484f00a08630740010d889/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:11.436632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df89f4-f62d-4108-af09-0024a2317761 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.436930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df89f4-f62d-4108-af09-0024a2317761 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.437263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df89f4-f62d-4108-af09-0024a2317761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.463172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b4df89f4-f62d-4108-af09-0024a2317761 None None] MFA Rules not processed for user `d672ced33c484f00a08630740010d889`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:11.464875 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b4df89f4-f62d-4108-af09-0024a2317761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:11.489371 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:11.531681 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1320/2639] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:11.543113 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b90a60-5e51-4ca4-84e9-b8033c1e1f1f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:11.544837 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e2b90a60-5e51-4ca4-84e9-b8033c1e1f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:11.602102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b90a60-5e51-4ca4-84e9-b8033c1e1f1f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:11.605849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1198e84-e42a-4bbb-b532-f2f082bb76a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:11.606493 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1198e84-e42a-4bbb-b532-f2f082bb76a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:11.606780 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1198e84-e42a-4bbb-b532-f2f082bb76a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:11.607062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1198e84-e42a-4bbb-b532-f2f082bb76a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:11.615616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1198e84-e42a-4bbb-b532-f2f082bb76a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:11.617049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1198e84-e42a-4bbb-b532-f2f082bb76a0 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:11.626191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1320/2640] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:12.413598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df89f4-f62d-4108-af09-0024a2317761 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:12.415041 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b4df89f4-f62d-4108-af09-0024a2317761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:12.480150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df89f4-f62d-4108-af09-0024a2317761 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:12.484657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:12.485565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:12.485856 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:12.486165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] PATH_INFO: `/v3/users/e9df4b165c784bac914b0e91dfbe6e39` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:12.487387 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9df4b165c784bac914b0e91dfbe6e39)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:12.489344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:12.535711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9df4b165c784bac914b0e91dfbe6e39'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:12.547879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9df4b165c784bac914b0e91dfbe6e39'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:12.555190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9df4b165c784bac914b0e91dfbe6e39'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:12.562716 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9df4b165c784bac914b0e91dfbe6e39'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:12.563355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9df4b165c784bac914b0e91dfbe6e39'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:12.569705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1321/2641] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:12 2022] DELETE /identity/v3/users/e9df4b165c784bac914b0e91dfbe6e39 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:12.574211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a37528-6a44-4ae3-98cb-77e77ec90e46 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:12.574691 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a37528-6a44-4ae3-98cb-77e77ec90e46 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:12.574808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a37528-6a44-4ae3-98cb-77e77ec90e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:12.592392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f2a37528-6a44-4ae3-98cb-77e77ec90e46 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:12.593646 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f2a37528-6a44-4ae3-98cb-77e77ec90e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:12.656829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1321/2642] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:12.666632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:12.668639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:12.717075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:12.728692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ba1d5-e675-4038-a75d-943f5b51ce40 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:12.732386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89ed04-8b6e-4f50-bf4e-598adde1492d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:12.733036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89ed04-8b6e-4f50-bf4e-598adde1492d service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:12.733350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89ed04-8b6e-4f50-bf4e-598adde1492d service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:12.733630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89ed04-8b6e-4f50-bf4e-598adde1492d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:12.743018 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89ed04-8b6e-4f50-bf4e-598adde1492d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:12.744509 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89ed04-8b6e-4f50-bf4e-598adde1492d service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:12.754562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1322/2643] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:13.029927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a37528-6a44-4ae3-98cb-77e77ec90e46 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:13.031528 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f2a37528-6a44-4ae3-98cb-77e77ec90e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:13.046511 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a37528-6a44-4ae3-98cb-77e77ec90e46 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:13.051737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:13.052611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:13.052898 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:13.053206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] PATH_INFO: `/v3/projects/8551c5bb991149b3a85cdf23af581efa` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:13.054412 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8551c5bb991149b3a85cdf23af581efa)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:13.057191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:13.105818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8551c5bb991149b3a85cdf23af581efa', 'request_id': 'req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.112841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8551c5bb991149b3a85cdf23af581efa', 'request_id': 'req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.122160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8551c5bb991149b3a85cdf23af581efa', 'request_id': 'req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.134826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] The token cache is being invalidate because project 8551c5bb991149b3a85cdf23af581efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:13.135292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.135974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.137581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1322/2644] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:13 2022] DELETE /identity/v3/projects/8551c5bb991149b3a85cdf23af581efa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:13.141794 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89ed04-8b6e-4f50-bf4e-598adde1492d service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:13.151696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89ed04-8b6e-4f50-bf4e-598adde1492d service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:13.210810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:13.211721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:13.211995 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:13.212278 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] PATH_INFO: `/v3/users/54cf0f55aa6945039b281862ba7c37f1` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:13.213519 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54cf0f55aa6945039b281862ba7c37f1)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:13.216010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:13.268504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54cf0f55aa6945039b281862ba7c37f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.281911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54cf0f55aa6945039b281862ba7c37f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.290231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54cf0f55aa6945039b281862ba7c37f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.297734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54cf0f55aa6945039b281862ba7c37f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.298412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c3d151b1-99e6-4feb-b30e-55848498f6c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54cf0f55aa6945039b281862ba7c37f1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.304832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1323/2645] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:13 2022] DELETE /identity/v3/users/54cf0f55aa6945039b281862ba7c37f1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:13.593206 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:13.595623 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:13.622591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaa11a-e3ad-4e67-ad8a-c99bff27eb6b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:13.648465 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:13.683912 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:13.684696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:13.684984 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:13.685327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] PATH_INFO: `/v3/projects/bc9bfc9354a146af9e4ddc3cee8957e6` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:13.686494 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc9bfc9354a146af9e4ddc3cee8957e6)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:13.688400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:13.737209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc9bfc9354a146af9e4ddc3cee8957e6', 'request_id': 'req-5b37b122-5ecc-4971-9670-62dd5a5cddd4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.743677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc9bfc9354a146af9e4ddc3cee8957e6', 'request_id': 'req-5b37b122-5ecc-4971-9670-62dd5a5cddd4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.751939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc9bfc9354a146af9e4ddc3cee8957e6', 'request_id': 'req-5b37b122-5ecc-4971-9670-62dd5a5cddd4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.764085 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] The token cache is being invalidate because project bc9bfc9354a146af9e4ddc3cee8957e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:13.764369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.765143 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b37b122-5ecc-4971-9670-62dd5a5cddd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:13.766670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1323/2646] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:13 2022] DELETE /identity/v3/projects/bc9bfc9354a146af9e4ddc3cee8957e6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:13.772367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a88a4-d66c-4391-8947-359e19edb977 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:13.772367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a88a4-d66c-4391-8947-359e19edb977 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:13.772367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a88a4-d66c-4391-8947-359e19edb977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:13.795717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-706a88a4-d66c-4391-8947-359e19edb977 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:13.796984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-706a88a4-d66c-4391-8947-359e19edb977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:13.856479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1324/2647] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:13.864506 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4268ba1-6051-4d97-b21d-9d60a82347aa None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:13.864805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4268ba1-6051-4d97-b21d-9d60a82347aa None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:13.865114 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4268ba1-6051-4d97-b21d-9d60a82347aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:13.883585 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e4268ba1-6051-4d97-b21d-9d60a82347aa None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:13.884805 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e4268ba1-6051-4d97-b21d-9d60a82347aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:13.894217 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1324/2648] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:13.898567 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a88a4-d66c-4391-8947-359e19edb977 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:13.909158 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a88a4-d66c-4391-8947-359e19edb977 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:13.914037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697735dc-4d02-4618-8546-8e6fe1e0aaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:13.914743 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697735dc-4d02-4618-8546-8e6fe1e0aaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:13.915066 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697735dc-4d02-4618-8546-8e6fe1e0aaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:13.915272 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697735dc-4d02-4618-8546-8e6fe1e0aaeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:13.916011 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697735dc-4d02-4618-8546-8e6fe1e0aaeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:15:13.916325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697735dc-4d02-4618-8546-8e6fe1e0aaeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:13.917965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697735dc-4d02-4618-8546-8e6fe1e0aaeb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:13.919563 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:13.932324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1325/2649] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:15:13.936678 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4268ba1-6051-4d97-b21d-9d60a82347aa None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:13.946276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4268ba1-6051-4d97-b21d-9d60a82347aa None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:13.952434 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd18bc-843c-4d35-bc54-a9f7ed0143a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:13.953115 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd18bc-843c-4d35-bc54-a9f7ed0143a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:13.953421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd18bc-843c-4d35-bc54-a9f7ed0143a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:13.953694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd18bc-843c-4d35-bc54-a9f7ed0143a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:13.954405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd18bc-843c-4d35-bc54-a9f7ed0143a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:13.956302 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd18bc-843c-4d35-bc54-a9f7ed0143a7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:13.982151 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1325/2650] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:13 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:13.986329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697735dc-4d02-4618-8546-8e6fe1e0aaeb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:13.996266 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697735dc-4d02-4618-8546-8e6fe1e0aaeb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.052088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb055891-2576-4ba6-8ab6-7db8c2d81453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.052779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb055891-2576-4ba6-8ab6-7db8c2d81453 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.053059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb055891-2576-4ba6-8ab6-7db8c2d81453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.053370 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb055891-2576-4ba6-8ab6-7db8c2d81453 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.054124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb055891-2576-4ba6-8ab6-7db8c2d81453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.055958 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb055891-2576-4ba6-8ab6-7db8c2d81453 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.076358 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1326/2651] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:13 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:14.080376 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd18bc-843c-4d35-bc54-a9f7ed0143a7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.091735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd18bc-843c-4d35-bc54-a9f7ed0143a7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.096679 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e8ca0-d228-4887-992d-02231b89a44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.097373 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e8ca0-d228-4887-992d-02231b89a44e admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.097664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e8ca0-d228-4887-992d-02231b89a44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.097946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e8ca0-d228-4887-992d-02231b89a44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.098673 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e8ca0-d228-4887-992d-02231b89a44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.100276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e8ca0-d228-4887-992d-02231b89a44e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.110878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1326/2652] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:14.114833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb055891-2576-4ba6-8ab6-7db8c2d81453 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.125736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb055891-2576-4ba6-8ab6-7db8c2d81453 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.132092 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369562b3-9d4c-449e-8edf-6c57e91c7dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.132770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369562b3-9d4c-449e-8edf-6c57e91c7dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.133063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369562b3-9d4c-449e-8edf-6c57e91c7dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.133393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369562b3-9d4c-449e-8edf-6c57e91c7dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.134121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369562b3-9d4c-449e-8edf-6c57e91c7dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.136336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369562b3-9d4c-449e-8edf-6c57e91c7dbe admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.147042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1327/2653] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:14.151255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e8ca0-d228-4887-992d-02231b89a44e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.161546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e8ca0-d228-4887-992d-02231b89a44e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.166194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6465b3f-bc44-40fc-9bc1-8c6d97d59acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.167096 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465b3f-bc44-40fc-9bc1-8c6d97d59acf admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.167383 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465b3f-bc44-40fc-9bc1-8c6d97d59acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.167667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465b3f-bc44-40fc-9bc1-8c6d97d59acf admin admin] PATH_INFO: `/v3/projects/953f2f092aec49c5892e9d450d8046c4/users/88bc0e31a88145ccab1c31520ef144d1/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.183784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6465b3f-bc44-40fc-9bc1-8c6d97d59acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=953f2f092aec49c5892e9d450d8046c4, user_id=88bc0e31a88145ccab1c31520ef144d1, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.186447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6465b3f-bc44-40fc-9bc1-8c6d97d59acf admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.200108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1327/2654] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:14 2022] PUT /identity/v3/projects/953f2f092aec49c5892e9d450d8046c4/users/88bc0e31a88145ccab1c31520ef144d1/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:14.210314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c5688-6584-4b5e-8724-bfbef5146f69 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.210636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c5688-6584-4b5e-8724-bfbef5146f69 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.210917 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c5688-6584-4b5e-8724-bfbef5146f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.237118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-762c5688-6584-4b5e-8724-bfbef5146f69 None None] MFA Rules not processed for user `88bc0e31a88145ccab1c31520ef144d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:14.238365 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-762c5688-6584-4b5e-8724-bfbef5146f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:14.302824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1328/2655] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:14.308080 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6465b3f-bc44-40fc-9bc1-8c6d97d59acf admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.363079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:14.375396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6465b3f-bc44-40fc-9bc1-8c6d97d59acf admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.380468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bfa62-631a-4683-9bcf-27e7b4dcf92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.381150 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bfa62-631a-4683-9bcf-27e7b4dcf92d admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.381440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bfa62-631a-4683-9bcf-27e7b4dcf92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.381720 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bfa62-631a-4683-9bcf-27e7b4dcf92d admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.382415 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bfa62-631a-4683-9bcf-27e7b4dcf92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.384440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bfa62-631a-4683-9bcf-27e7b4dcf92d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.409386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1328/2656] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:14 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:14.413632 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c5688-6584-4b5e-8724-bfbef5146f69 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.424311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c5688-6584-4b5e-8724-bfbef5146f69 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.486243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e034eac-55b9-443d-9b1f-d112d869e734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.486819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e034eac-55b9-443d-9b1f-d112d869e734 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.487109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e034eac-55b9-443d-9b1f-d112d869e734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.487386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e034eac-55b9-443d-9b1f-d112d869e734 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.488068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e034eac-55b9-443d-9b1f-d112d869e734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.490010 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e034eac-55b9-443d-9b1f-d112d869e734 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.498044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:14.510203 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1329/2657] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:14 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:15:14.514089 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bfa62-631a-4683-9bcf-27e7b4dcf92d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.524404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bfa62-631a-4683-9bcf-27e7b4dcf92d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.529012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978d8b8-aac6-4f2d-bd81-784de9aa542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.529696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d8b8-aac6-4f2d-bd81-784de9aa542f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.529983 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d8b8-aac6-4f2d-bd81-784de9aa542f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.530261 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d8b8-aac6-4f2d-bd81-784de9aa542f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.530978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978d8b8-aac6-4f2d-bd81-784de9aa542f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.532588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978d8b8-aac6-4f2d-bd81-784de9aa542f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.543555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1329/2658] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:14.547667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e034eac-55b9-443d-9b1f-d112d869e734 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.558367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e034eac-55b9-443d-9b1f-d112d869e734 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.562991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837a09a-83d8-4cc7-b00f-00ab91cd0e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.563858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837a09a-83d8-4cc7-b00f-00ab91cd0e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.564150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837a09a-83d8-4cc7-b00f-00ab91cd0e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.564427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837a09a-83d8-4cc7-b00f-00ab91cd0e9f admin admin] PATH_INFO: `/v3/projects/f608b053a27745348a038586329416be/users/b65b5092b2aa4f6c8d8e671ff6861ae8/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.579227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837a09a-83d8-4cc7-b00f-00ab91cd0e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f608b053a27745348a038586329416be, user_id=b65b5092b2aa4f6c8d8e671ff6861ae8, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.582185 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837a09a-83d8-4cc7-b00f-00ab91cd0e9f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.595173 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1330/2659] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:14 2022] PUT /identity/v3/projects/f608b053a27745348a038586329416be/users/b65b5092b2aa4f6c8d8e671ff6861ae8/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:14.599233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978d8b8-aac6-4f2d-bd81-784de9aa542f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.610109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978d8b8-aac6-4f2d-bd81-784de9aa542f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.661463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837a09a-83d8-4cc7-b00f-00ab91cd0e9f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.663295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6837a09a-83d8-4cc7-b00f-00ab91cd0e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:14.669181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ba502-e9a5-4c71-a6ff-395a07bb06ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.669844 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ba502-e9a5-4c71-a6ff-395a07bb06ab admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.670128 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ba502-e9a5-4c71-a6ff-395a07bb06ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.670404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ba502-e9a5-4c71-a6ff-395a07bb06ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.671123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ba502-e9a5-4c71-a6ff-395a07bb06ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.672708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ba502-e9a5-4c71-a6ff-395a07bb06ab admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.677231 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837a09a-83d8-4cc7-b00f-00ab91cd0e9f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.682181 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.683126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1330/2660] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:14.683831 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.684122 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.684427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] PATH_INFO: `/v3/users/d672ced33c484f00a08630740010d889` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.685728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d672ced33c484f00a08630740010d889)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.687229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ba502-e9a5-4c71-a6ff-395a07bb06ab admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.687952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.697337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ba502-e9a5-4c71-a6ff-395a07bb06ab admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.702068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83321d-ada3-498e-b525-d48948f75540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.702945 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83321d-ada3-498e-b525-d48948f75540 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.703233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83321d-ada3-498e-b525-d48948f75540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.703659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83321d-ada3-498e-b525-d48948f75540 admin admin] PATH_INFO: `/v3/projects/f608b053a27745348a038586329416be/users/b65b5092b2aa4f6c8d8e671ff6861ae8/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.705736 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83321d-ada3-498e-b525-d48948f75540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f608b053a27745348a038586329416be, user_id=b65b5092b2aa4f6c8d8e671ff6861ae8, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.708350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83321d-ada3-498e-b525-d48948f75540 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.720472 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1331/2661] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:14 2022] PUT /identity/v3/projects/f608b053a27745348a038586329416be/users/b65b5092b2aa4f6c8d8e671ff6861ae8/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:15:14.732447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cd36e-ff2a-43c2-a32b-774c058b156d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.732746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cd36e-ff2a-43c2-a32b-774c058b156d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.733032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cd36e-ff2a-43c2-a32b-774c058b156d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.738890 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd672ced33c484f00a08630740010d889'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:14.752417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd672ced33c484f00a08630740010d889'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:14.760166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd672ced33c484f00a08630740010d889'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:14.760725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9b2cd36e-ff2a-43c2-a32b-774c058b156d None None] MFA Rules not processed for user `b65b5092b2aa4f6c8d8e671ff6861ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:14.762174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9b2cd36e-ff2a-43c2-a32b-774c058b156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:14.767493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd672ced33c484f00a08630740010d889'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:14.768263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba003e53-f3d4-4bc1-988d-b134271c3f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd672ced33c484f00a08630740010d889'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:14.774328 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1331/2662] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:14 2022] DELETE /identity/v3/users/d672ced33c484f00a08630740010d889 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:14.778720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e04cc-38b6-45d5-8781-a5aed860edec None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.779064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e04cc-38b6-45d5-8781-a5aed860edec None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.779346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e04cc-38b6-45d5-8781-a5aed860edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.797308 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-418e04cc-38b6-45d5-8781-a5aed860edec None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:14.798543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-418e04cc-38b6-45d5-8781-a5aed860edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:14.834827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1332/2663] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:14.846303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cd36e-ff2a-43c2-a32b-774c058b156d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.847873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9b2cd36e-ff2a-43c2-a32b-774c058b156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:14.860037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1332/2664] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:14.870492 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e04cc-38b6-45d5-8781-a5aed860edec None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:14.872051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-418e04cc-38b6-45d5-8781-a5aed860edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:14.890170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:14.909433 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cd36e-ff2a-43c2-a32b-774c058b156d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.915892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:14.932438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e04cc-38b6-45d5-8781-a5aed860edec None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:14.936318 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93107a0-b968-4cde-9dc3-5b785a4d32ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.937063 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93107a0-b968-4cde-9dc3-5b785a4d32ee service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.941581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93107a0-b968-4cde-9dc3-5b785a4d32ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.941877 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93107a0-b968-4cde-9dc3-5b785a4d32ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.959723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93107a0-b968-4cde-9dc3-5b785a4d32ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:14.959947 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96208ad5-1276-494f-b8c7-d314c88b7e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:14.960392 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96208ad5-1276-494f-b8c7-d314c88b7e59 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:14.960677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96208ad5-1276-494f-b8c7-d314c88b7e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:14.960956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96208ad5-1276-494f-b8c7-d314c88b7e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:14.961186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93107a0-b968-4cde-9dc3-5b785a4d32ee service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:14.976507 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1333/2665] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:15:15.024663 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96208ad5-1276-494f-b8c7-d314c88b7e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:15.026901 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96208ad5-1276-494f-b8c7-d314c88b7e59 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:15.037538 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1333/2666] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:15:15.143972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93107a0-b968-4cde-9dc3-5b785a4d32ee service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:15.154691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93107a0-b968-4cde-9dc3-5b785a4d32ee service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:15.158536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff234c-b2b7-4018-b1d7-bcbbb2c8bc9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:15.159183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff234c-b2b7-4018-b1d7-bcbbb2c8bc9e service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.159467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff234c-b2b7-4018-b1d7-bcbbb2c8bc9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.160177 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff234c-b2b7-4018-b1d7-bcbbb2c8bc9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.161599 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-84ff234c-b2b7-4018-b1d7-bcbbb2c8bc9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:15.250790 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff234c-b2b7-4018-b1d7-bcbbb2c8bc9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:15.252526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff234c-b2b7-4018-b1d7-bcbbb2c8bc9e service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:15.264853 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1334/2667] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:15.279495 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96208ad5-1276-494f-b8c7-d314c88b7e59 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:15.281204 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-96208ad5-1276-494f-b8c7-d314c88b7e59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:15.328297 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96208ad5-1276-494f-b8c7-d314c88b7e59 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:15.334482 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:15.335348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.335751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.336030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] PATH_INFO: `/v3/projects/b98858235df741bab2a3fb91a470df63` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.337249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b98858235df741bab2a3fb91a470df63)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:15.339778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:15.390243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b98858235df741bab2a3fb91a470df63', 'request_id': 'req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:15.397229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b98858235df741bab2a3fb91a470df63', 'request_id': 'req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:15.406309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b98858235df741bab2a3fb91a470df63', 'request_id': 'req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:15.417756 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] The token cache is being invalidate because project b98858235df741bab2a3fb91a470df63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:15.418136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:15.418874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-85942956-f7f6-4c4d-a8af-38f5ae13ebe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:15.420629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1334/2668] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:15 2022] DELETE /identity/v3/projects/b98858235df741bab2a3fb91a470df63 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:15.426774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c6ae4-6b48-4606-bbce-c6bf346bd21c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.426934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c6ae4-6b48-4606-bbce-c6bf346bd21c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.427215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c6ae4-6b48-4606-bbce-c6bf346bd21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.447614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6d5c6ae4-6b48-4606-bbce-c6bf346bd21c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:15.449401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6d5c6ae4-6b48-4606-bbce-c6bf346bd21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:15.514338 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1335/2669] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:15.525667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41654-0877-4496-a26e-563b3ab72799 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.526644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41654-0877-4496-a26e-563b3ab72799 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.526979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41654-0877-4496-a26e-563b3ab72799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.545779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1aa41654-0877-4496-a26e-563b3ab72799 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:15.546969 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1aa41654-0877-4496-a26e-563b3ab72799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:15.557356 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1335/2670] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:15.560013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c6ae4-6b48-4606-bbce-c6bf346bd21c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:15.575482 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c6ae4-6b48-4606-bbce-c6bf346bd21c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:15.580696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d1d8-fac7-4f73-88a6-780987ef36ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:15.581332 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d1d8-fac7-4f73-88a6-780987ef36ae admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.581624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d1d8-fac7-4f73-88a6-780987ef36ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.581892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d1d8-fac7-4f73-88a6-780987ef36ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.582876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d1d8-fac7-4f73-88a6-780987ef36ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:15:15.583298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d1d8-fac7-4f73-88a6-780987ef36ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:15.585007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364d1d8-fac7-4f73-88a6-780987ef36ae admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:15.598126 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1336/2671] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:15:15.603106 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41654-0877-4496-a26e-563b3ab72799 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:15.609885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:15.616414 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41654-0877-4496-a26e-563b3ab72799 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:15.622951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e56780-9646-4c72-9f89-c05ef98e6077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:15.623605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e56780-9646-4c72-9f89-c05ef98e6077 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.623907 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e56780-9646-4c72-9f89-c05ef98e6077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.624173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e56780-9646-4c72-9f89-c05ef98e6077 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.624868 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e56780-9646-4c72-9f89-c05ef98e6077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:15.626872 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e56780-9646-4c72-9f89-c05ef98e6077 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:15.653684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1336/2672] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:15 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:15.657662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d1d8-fac7-4f73-88a6-780987ef36ae admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:15.667813 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d1d8-fac7-4f73-88a6-780987ef36ae admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:15.713046 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:15.729408 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3b74b-342c-474d-b44f-91c52076a18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:15.730099 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3b74b-342c-474d-b44f-91c52076a18b admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.730382 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3b74b-342c-474d-b44f-91c52076a18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.730657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3b74b-342c-474d-b44f-91c52076a18b admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.731341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3b74b-342c-474d-b44f-91c52076a18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:15.733437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3b74b-342c-474d-b44f-91c52076a18b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:15.753043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1337/2673] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:15 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:15.757264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e56780-9646-4c72-9f89-c05ef98e6077 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:15.771390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e56780-9646-4c72-9f89-c05ef98e6077 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:15.784558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae775d-c972-467e-9651-4c25eba745fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:15.785181 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae775d-c972-467e-9651-4c25eba745fe admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.785466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae775d-c972-467e-9651-4c25eba745fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.785743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae775d-c972-467e-9651-4c25eba745fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.786462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae775d-c972-467e-9651-4c25eba745fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:15.788041 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae775d-c972-467e-9651-4c25eba745fe admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:15.798205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1337/2674] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:15:15.802224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3b74b-342c-474d-b44f-91c52076a18b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:15.830534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3b74b-342c-474d-b44f-91c52076a18b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:15.835178 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278891ed-6cee-42ba-a212-9f2a1a79967a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:15.835852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278891ed-6cee-42ba-a212-9f2a1a79967a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.836144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278891ed-6cee-42ba-a212-9f2a1a79967a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.836429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278891ed-6cee-42ba-a212-9f2a1a79967a admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.837226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278891ed-6cee-42ba-a212-9f2a1a79967a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:15.838905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278891ed-6cee-42ba-a212-9f2a1a79967a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:15.849573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1338/2675] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:15.853785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae775d-c972-467e-9651-4c25eba745fe admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:15.863705 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae775d-c972-467e-9651-4c25eba745fe admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:15.868215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67680520-288a-4d0d-ad19-3edc7ec55254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:15.869075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67680520-288a-4d0d-ad19-3edc7ec55254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.869388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67680520-288a-4d0d-ad19-3edc7ec55254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.869661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67680520-288a-4d0d-ad19-3edc7ec55254 admin admin] PATH_INFO: `/v3/projects/71b918ee36fa4888a6d708cfac6c2995/users/2bd39a103c444535b0bf643f45d91945/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.884501 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67680520-288a-4d0d-ad19-3edc7ec55254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b918ee36fa4888a6d708cfac6c2995, user_id=2bd39a103c444535b0bf643f45d91945, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:15.887810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67680520-288a-4d0d-ad19-3edc7ec55254 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:15.900560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1338/2676] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:15 2022] PUT /identity/v3/projects/71b918ee36fa4888a6d708cfac6c2995/users/2bd39a103c444535b0bf643f45d91945/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:15.910873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a9887-cb60-4f18-b192-4271e2a15a05 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:15.911170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a9887-cb60-4f18-b192-4271e2a15a05 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:15.911454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a9887-cb60-4f18-b192-4271e2a15a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:15.939121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5c5a9887-cb60-4f18-b192-4271e2a15a05 None None] MFA Rules not processed for user `2bd39a103c444535b0bf643f45d91945`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:15.940336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5c5a9887-cb60-4f18-b192-4271e2a15a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:16.005107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1339/2677] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:16.016180 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67680520-288a-4d0d-ad19-3edc7ec55254 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:16.017911 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-67680520-288a-4d0d-ad19-3edc7ec55254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:16.080703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67680520-288a-4d0d-ad19-3edc7ec55254 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:16.084358 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6557f-5a35-47b8-91ea-4db9942256c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:16.084994 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6557f-5a35-47b8-91ea-4db9942256c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:16.085319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6557f-5a35-47b8-91ea-4db9942256c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:16.085601 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6557f-5a35-47b8-91ea-4db9942256c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:16.093993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6557f-5a35-47b8-91ea-4db9942256c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:16.095417 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6557f-5a35-47b8-91ea-4db9942256c6 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:16.104833 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1339/2678] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:17.499412 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd9c5d-6f74-4e56-abd0-af5f47f9ac83 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:17.499999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd9c5d-6f74-4e56-abd0-af5f47f9ac83 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:17.499999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd9c5d-6f74-4e56-abd0-af5f47f9ac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:17.517962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-60dd9c5d-6f74-4e56-abd0-af5f47f9ac83 None None] MFA Rules not processed for user `2bd39a103c444535b0bf643f45d91945`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:17.519171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-60dd9c5d-6f74-4e56-abd0-af5f47f9ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:17.526356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1340/2679] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:17.855699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6557f-5a35-47b8-91ea-4db9942256c6 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:17.857337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bbc6557f-5a35-47b8-91ea-4db9942256c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:17.925119 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6557f-5a35-47b8-91ea-4db9942256c6 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:17.929619 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:17.930488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:17.930774 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:17.931061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] PATH_INFO: `/v3/users/88bc0e31a88145ccab1c31520ef144d1` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:17.932249 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=88bc0e31a88145ccab1c31520ef144d1)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:17.934735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:17.983762 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:17.996055 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd9c5d-6f74-4e56-abd0-af5f47f9ac83 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:17.998696 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-60dd9c5d-6f74-4e56-abd0-af5f47f9ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:18.001270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88bc0e31a88145ccab1c31520ef144d1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.014871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88bc0e31a88145ccab1c31520ef144d1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.024309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88bc0e31a88145ccab1c31520ef144d1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.024954 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88bc0e31a88145ccab1c31520ef144d1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.032993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dc0b1fb1-114b-4d31-af30-e1945243fffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88bc0e31a88145ccab1c31520ef144d1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.040590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1340/2680] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:17 2022] DELETE /identity/v3/users/88bc0e31a88145ccab1c31520ef144d1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:18.045019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88104b05-f7fa-4896-a44d-7e54a0d8596b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:18.045483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88104b05-f7fa-4896-a44d-7e54a0d8596b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:18.045825 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88104b05-f7fa-4896-a44d-7e54a0d8596b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:18.065624 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-88104b05-f7fa-4896-a44d-7e54a0d8596b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:18.066846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-88104b05-f7fa-4896-a44d-7e54a0d8596b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:18.083989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd9c5d-6f74-4e56-abd0-af5f47f9ac83 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:18.123195 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:18.127374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1341/2681] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:18.136839 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88104b05-f7fa-4896-a44d-7e54a0d8596b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:18.138886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-88104b05-f7fa-4896-a44d-7e54a0d8596b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:18.150502 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:18.151190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:18.151529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:18.151819 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:18.153951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:18.166459 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:18.168064 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:18.178469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1341/2682] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:18.198929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88104b05-f7fa-4896-a44d-7e54a0d8596b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:18.203109 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe97bc-3ecc-4fd5-9991-fabbf1d0b05d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:18.203750 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe97bc-3ecc-4fd5-9991-fabbf1d0b05d service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:18.204035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe97bc-3ecc-4fd5-9991-fabbf1d0b05d service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:18.204313 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe97bc-3ecc-4fd5-9991-fabbf1d0b05d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:18.215990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe97bc-3ecc-4fd5-9991-fabbf1d0b05d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:18.218536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe97bc-3ecc-4fd5-9991-fabbf1d0b05d service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:18.229566 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1342/2683] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:15:18.513242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:18.515368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:18.540277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3914eaa-d029-43b2-bc9e-0b77f5545b59 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:18.545443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:18.546307 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:18.546596 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:18.546878 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] PATH_INFO: `/v3/projects/953f2f092aec49c5892e9d450d8046c4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:18.548977 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=953f2f092aec49c5892e9d450d8046c4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:18.550914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:18.601961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953f2f092aec49c5892e9d450d8046c4', 'request_id': 'req-f8ace269-cce3-4b78-aa5b-568bddf956ea'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.609168 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953f2f092aec49c5892e9d450d8046c4', 'request_id': 'req-f8ace269-cce3-4b78-aa5b-568bddf956ea'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.618428 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953f2f092aec49c5892e9d450d8046c4', 'request_id': 'req-f8ace269-cce3-4b78-aa5b-568bddf956ea'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.631074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] The token cache is being invalidate because project 953f2f092aec49c5892e9d450d8046c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:18.631372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.632047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.633989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1342/2684] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:18 2022] DELETE /identity/v3/projects/953f2f092aec49c5892e9d450d8046c4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:18.638557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe97bc-3ecc-4fd5-9991-fabbf1d0b05d service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:18.649924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe97bc-3ecc-4fd5-9991-fabbf1d0b05d service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:18.701437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:18.709398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:18.710326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:18.710785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:18.711327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] PATH_INFO: `/v3/users/b65b5092b2aa4f6c8d8e671ff6861ae8` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:18.712693 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65b5092b2aa4f6c8d8e671ff6861ae8)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:18.714641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:18.763978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65b5092b2aa4f6c8d8e671ff6861ae8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.777345 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65b5092b2aa4f6c8d8e671ff6861ae8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.785797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65b5092b2aa4f6c8d8e671ff6861ae8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.786570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65b5092b2aa4f6c8d8e671ff6861ae8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.794232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7a923036-676d-4cef-adf5-4367079dfaee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65b5092b2aa4f6c8d8e671ff6861ae8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:18.800172 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1343/2685] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:18 2022] DELETE /identity/v3/users/b65b5092b2aa4f6c8d8e671ff6861ae8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:19.081166 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:19.082803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:19.101181 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ace269-cce3-4b78-aa5b-568bddf956ea admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:19.196736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:19.197653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.197937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.198220 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] PATH_INFO: `/v3/projects/f608b053a27745348a038586329416be` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.199445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f608b053a27745348a038586329416be)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:19.201442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:19.261397 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f608b053a27745348a038586329416be', 'request_id': 'req-ed562cf5-60ba-42c0-9143-f2177b02bde6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:19.269658 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f608b053a27745348a038586329416be', 'request_id': 'req-ed562cf5-60ba-42c0-9143-f2177b02bde6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:19.275460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:19.278304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f608b053a27745348a038586329416be', 'request_id': 'req-ed562cf5-60ba-42c0-9143-f2177b02bde6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:19.291172 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] The token cache is being invalidate because project f608b053a27745348a038586329416be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:19.291467 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:19.292104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ed562cf5-60ba-42c0-9143-f2177b02bde6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:19.294211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1343/2686] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:19 2022] DELETE /identity/v3/projects/f608b053a27745348a038586329416be => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:19.299795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cead5fa-bcd1-4360-8671-402fe6416c36 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.300265 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cead5fa-bcd1-4360-8671-402fe6416c36 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.300390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cead5fa-bcd1-4360-8671-402fe6416c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.319652 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6cead5fa-bcd1-4360-8671-402fe6416c36 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:19.320918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6cead5fa-bcd1-4360-8671-402fe6416c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:19.382168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1344/2687] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:19.390050 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ef868-bba9-4333-9329-269a5847c908 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.390344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ef868-bba9-4333-9329-269a5847c908 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.390629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ef868-bba9-4333-9329-269a5847c908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.409390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-571ef868-bba9-4333-9329-269a5847c908 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:19.410612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-571ef868-bba9-4333-9329-269a5847c908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:19.419320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1344/2688] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:19.423549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cead5fa-bcd1-4360-8671-402fe6416c36 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:19.434629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cead5fa-bcd1-4360-8671-402fe6416c36 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:19.439248 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee993749-4c7d-4bce-9044-54be77e94a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:19.439917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee993749-4c7d-4bce-9044-54be77e94a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.440199 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee993749-4c7d-4bce-9044-54be77e94a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.440477 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee993749-4c7d-4bce-9044-54be77e94a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.441394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee993749-4c7d-4bce-9044-54be77e94a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:15:19.441707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee993749-4c7d-4bce-9044-54be77e94a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:19.443321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee993749-4c7d-4bce-9044-54be77e94a22 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:19.456378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1345/2689] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:15:19.460454 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ef868-bba9-4333-9329-269a5847c908 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:19.470586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ef868-bba9-4333-9329-269a5847c908 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:19.475346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c4dd9-81a5-46d5-b4d8-6c8fe7b73fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:19.476007 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c4dd9-81a5-46d5-b4d8-6c8fe7b73fee admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.476291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c4dd9-81a5-46d5-b4d8-6c8fe7b73fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.476569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c4dd9-81a5-46d5-b4d8-6c8fe7b73fee admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.477341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c4dd9-81a5-46d5-b4d8-6c8fe7b73fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:19.479250 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c4dd9-81a5-46d5-b4d8-6c8fe7b73fee admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:19.507633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1345/2690] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:19 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:19.511738 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee993749-4c7d-4bce-9044-54be77e94a22 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:19.524295 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee993749-4c7d-4bce-9044-54be77e94a22 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:19.586721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f003cb-a97a-499a-aee1-91e5b70247c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:19.587674 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f003cb-a97a-499a-aee1-91e5b70247c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.587961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f003cb-a97a-499a-aee1-91e5b70247c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.588243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f003cb-a97a-499a-aee1-91e5b70247c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.588916 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f003cb-a97a-499a-aee1-91e5b70247c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:19.591388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f003cb-a97a-499a-aee1-91e5b70247c6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:19.599830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:19.615296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1346/2691] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:19 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:19.619707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c4dd9-81a5-46d5-b4d8-6c8fe7b73fee admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:19.630544 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c4dd9-81a5-46d5-b4d8-6c8fe7b73fee admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:19.635271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235f0a8-f240-41df-a794-71c2a7d630ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:19.635938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235f0a8-f240-41df-a794-71c2a7d630ca admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.636221 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235f0a8-f240-41df-a794-71c2a7d630ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.636500 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235f0a8-f240-41df-a794-71c2a7d630ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.637331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235f0a8-f240-41df-a794-71c2a7d630ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:19.639102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235f0a8-f240-41df-a794-71c2a7d630ca admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:19.651062 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1346/2692] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:15:19.655198 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f003cb-a97a-499a-aee1-91e5b70247c6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:19.665478 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f003cb-a97a-499a-aee1-91e5b70247c6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:19.670790 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca2242-5b66-48fe-b56b-d3a0d82608d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:19.671419 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca2242-5b66-48fe-b56b-d3a0d82608d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.671703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca2242-5b66-48fe-b56b-d3a0d82608d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.671985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca2242-5b66-48fe-b56b-d3a0d82608d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.672707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca2242-5b66-48fe-b56b-d3a0d82608d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:19.674330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca2242-5b66-48fe-b56b-d3a0d82608d7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:19.684568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1347/2693] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:19.688629 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235f0a8-f240-41df-a794-71c2a7d630ca admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:19.699297 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235f0a8-f240-41df-a794-71c2a7d630ca admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:19.705218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc568a8f-851c-4a88-8c2c-e3e096d861d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:19.706138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc568a8f-851c-4a88-8c2c-e3e096d861d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.706424 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc568a8f-851c-4a88-8c2c-e3e096d861d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.706707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc568a8f-851c-4a88-8c2c-e3e096d861d6 admin admin] PATH_INFO: `/v3/projects/ba68bf382a1d43bd984e1e8fcb2c5d79/users/f743e5b5c832415b8c450463a8a005a3/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.724443 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc568a8f-851c-4a88-8c2c-e3e096d861d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba68bf382a1d43bd984e1e8fcb2c5d79, user_id=f743e5b5c832415b8c450463a8a005a3, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:19.727620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc568a8f-851c-4a88-8c2c-e3e096d861d6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:19.740896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1347/2694] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:19 2022] PUT /identity/v3/projects/ba68bf382a1d43bd984e1e8fcb2c5d79/users/f743e5b5c832415b8c450463a8a005a3/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:19.752223 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed011-e4ff-455d-a134-7a5664b57976 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.752561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed011-e4ff-455d-a134-7a5664b57976 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.752855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed011-e4ff-455d-a134-7a5664b57976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.781303 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3f4ed011-e4ff-455d-a134-7a5664b57976 None None] MFA Rules not processed for user `f743e5b5c832415b8c450463a8a005a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:19.782541 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3f4ed011-e4ff-455d-a134-7a5664b57976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:19.849276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1348/2695] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:19.861120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc568a8f-851c-4a88-8c2c-e3e096d861d6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:19.862955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bc568a8f-851c-4a88-8c2c-e3e096d861d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:19.926511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc568a8f-851c-4a88-8c2c-e3e096d861d6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:19.930148 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e9fb0-fad2-4f3f-a558-2b2b88a564ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:19.930800 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e9fb0-fad2-4f3f-a558-2b2b88a564ee service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:19.931088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e9fb0-fad2-4f3f-a558-2b2b88a564ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:19.931405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e9fb0-fad2-4f3f-a558-2b2b88a564ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:19.940968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e9fb0-fad2-4f3f-a558-2b2b88a564ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:19.942451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e9fb0-fad2-4f3f-a558-2b2b88a564ee service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:19.952723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1348/2696] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:19.973692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ed011-e4ff-455d-a134-7a5664b57976 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:20.053245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ed011-e4ff-455d-a134-7a5664b57976 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:20.058289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:20.059670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:20.061081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:20.061081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] PATH_INFO: `/v3/users/f743e5b5c832415b8c450463a8a005a3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:20.061719 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f743e5b5c832415b8c450463a8a005a3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:20.063723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:20.116763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f743e5b5c832415b8c450463a8a005a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.133597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f743e5b5c832415b8c450463a8a005a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.145943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f743e5b5c832415b8c450463a8a005a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.146567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f743e5b5c832415b8c450463a8a005a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.148215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:20.157818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f743e5b5c832415b8c450463a8a005a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.171153 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1349/2697] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:19 2022] DELETE /identity/v3/users/f743e5b5c832415b8c450463a8a005a3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:20.176121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc02216-c3a6-4c49-a53b-0d6c0c37f0ef None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:20.176411 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc02216-c3a6-4c49-a53b-0d6c0c37f0ef None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:20.176756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc02216-c3a6-4c49-a53b-0d6c0c37f0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:20.196493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9bc02216-c3a6-4c49-a53b-0d6c0c37f0ef None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:20.197795 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9bc02216-c3a6-4c49-a53b-0d6c0c37f0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:20.282848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1349/2698] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:20.294724 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:20.297802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:20.363214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8b0c6-a3ae-40af-b17b-9e260b32f0bb admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:20.367121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb0254-8d23-4d56-899f-b4ca47f2f56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:20.367775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0254-8d23-4d56-899f-b4ca47f2f56e service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:20.368084 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0254-8d23-4d56-899f-b4ca47f2f56e service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:20.368367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0254-8d23-4d56-899f-b4ca47f2f56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:20.378631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb0254-8d23-4d56-899f-b4ca47f2f56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:20.380162 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb0254-8d23-4d56-899f-b4ca47f2f56e service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:20.390927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1350/2699] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:15:20.685591 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc02216-c3a6-4c49-a53b-0d6c0c37f0ef None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:20.687581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9bc02216-c3a6-4c49-a53b-0d6c0c37f0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:20.710857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc02216-c3a6-4c49-a53b-0d6c0c37f0ef None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:20.721634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:20.722516 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:20.722797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:20.723073 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] PATH_INFO: `/v3/projects/ba68bf382a1d43bd984e1e8fcb2c5d79` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:20.724312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba68bf382a1d43bd984e1e8fcb2c5d79)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:20.726258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:20.747373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:20.782322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba68bf382a1d43bd984e1e8fcb2c5d79', 'request_id': 'req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.789045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba68bf382a1d43bd984e1e8fcb2c5d79', 'request_id': 'req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.798379 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba68bf382a1d43bd984e1e8fcb2c5d79', 'request_id': 'req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.815345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] The token cache is being invalidate because project ba68bf382a1d43bd984e1e8fcb2c5d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:20.816557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.817528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9badc3dc-da49-4b58-bdf0-ef1ed4dbcf50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:20.820042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1350/2700] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:20 2022] DELETE /identity/v3/projects/ba68bf382a1d43bd984e1e8fcb2c5d79 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:28.664926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb0254-8d23-4d56-899f-b4ca47f2f56e service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:28.666526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6bbb0254-8d23-4d56-899f-b4ca47f2f56e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:28.753866 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb0254-8d23-4d56-899f-b4ca47f2f56e service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:28.819238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:28.819795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:28.820082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:28.820361 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:28.822185 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:28.834228 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:28.835822 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:28.846571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1351/2701] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:30.808479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:34.977545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3fba4-1587-4733-80f2-28fdf0517c7a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:34.978112 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3fba4-1587-4733-80f2-28fdf0517c7a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:34.978112 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3fba4-1587-4733-80f2-28fdf0517c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:35.001636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2cc3fba4-1587-4733-80f2-28fdf0517c7a None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:35.002892 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2cc3fba4-1587-4733-80f2-28fdf0517c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:35.061111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1351/2702] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:15:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:35.106200 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:35.108326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:35.118167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:35.178442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c3da8-8c3b-4acb-9214-0002dc2552b0 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:35.183474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac345543-2647-4639-9016-5c8c89776392 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:35.184121 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac345543-2647-4639-9016-5c8c89776392 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:35.184410 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac345543-2647-4639-9016-5c8c89776392 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:35.184692 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac345543-2647-4639-9016-5c8c89776392 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:35.193240 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac345543-2647-4639-9016-5c8c89776392 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:35.195039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac345543-2647-4639-9016-5c8c89776392 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:35.203491 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1352/2703] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:15:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:15:35.713218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c309437-fd47-44a8-9801-00d4f763058d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:35.713520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c309437-fd47-44a8-9801-00d4f763058d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:35.713806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c309437-fd47-44a8-9801-00d4f763058d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:35.732386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3c309437-fd47-44a8-9801-00d4f763058d None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:35.733669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3c309437-fd47-44a8-9801-00d4f763058d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:35.740867 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1352/2704] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jun 21 19:15:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:35.753046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac345543-2647-4639-9016-5c8c89776392 service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:35.764320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac345543-2647-4639-9016-5c8c89776392 service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:35.769367 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799a3fc-8f80-474b-9381-ed5109092cb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:35.770010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799a3fc-8f80-474b-9381-ed5109092cb0 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:35.770296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799a3fc-8f80-474b-9381-ed5109092cb0 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:35.770581 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799a3fc-8f80-474b-9381-ed5109092cb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:35.779862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799a3fc-8f80-474b-9381-ed5109092cb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:35.781578 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799a3fc-8f80-474b-9381-ed5109092cb0 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:35.791254 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1353/2705] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:15:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:15:36.006504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77498e5-e2d4-4a96-a157-f004e63f109c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:36.007033 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77498e5-e2d4-4a96-a157-f004e63f109c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:36.007665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77498e5-e2d4-4a96-a157-f004e63f109c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:36.029770 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b77498e5-e2d4-4a96-a157-f004e63f109c None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:36.031655 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b77498e5-e2d4-4a96-a157-f004e63f109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:36.039262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1353/2706] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:15:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:36.052094 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799a3fc-8f80-474b-9381-ed5109092cb0 service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:36.063305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799a3fc-8f80-474b-9381-ed5109092cb0 service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:36.068648 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f03904-2c78-44e2-a515-86f32f971cbb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:36.069716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f03904-2c78-44e2-a515-86f32f971cbb service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:36.070075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f03904-2c78-44e2-a515-86f32f971cbb service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:36.070612 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f03904-2c78-44e2-a515-86f32f971cbb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:36.079875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f03904-2c78-44e2-a515-86f32f971cbb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:36.081821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f03904-2c78-44e2-a515-86f32f971cbb service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:36.089971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1354/2707] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 19:15:36.422444 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03958359-e9ae-43bb-a81a-baae3491ff45 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:36.422738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03958359-e9ae-43bb-a81a-baae3491ff45 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:36.423026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03958359-e9ae-43bb-a81a-baae3491ff45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:36.442251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-03958359-e9ae-43bb-a81a-baae3491ff45 None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:36.443392 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-03958359-e9ae-43bb-a81a-baae3491ff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:36.450876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1354/2708] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jun 21 19:15:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:36.462946 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f03904-2c78-44e2-a515-86f32f971cbb service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:36.473635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f03904-2c78-44e2-a515-86f32f971cbb service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:36.478731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19fcc9-f6ff-4911-85ff-9d70029fcfc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:36.479374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19fcc9-f6ff-4911-85ff-9d70029fcfc4 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:36.479658 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19fcc9-f6ff-4911-85ff-9d70029fcfc4 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:36.480048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19fcc9-f6ff-4911-85ff-9d70029fcfc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:36.489035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19fcc9-f6ff-4911-85ff-9d70029fcfc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:36.490635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19fcc9-f6ff-4911-85ff-9d70029fcfc4 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:36.499989 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1355/2709] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:15:38.555433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03958359-e9ae-43bb-a81a-baae3491ff45 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:38.556859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-03958359-e9ae-43bb-a81a-baae3491ff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:38.656165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03958359-e9ae-43bb-a81a-baae3491ff45 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:38.660755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:38.661702 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:38.661946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:38.662229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] PATH_INFO: `/v3/users/2bd39a103c444535b0bf643f45d91945` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:38.663438 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd39a103c444535b0bf643f45d91945)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:38.665410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:38.711693 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd39a103c444535b0bf643f45d91945'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:38.720252 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:38.725707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd39a103c444535b0bf643f45d91945'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:38.733593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd39a103c444535b0bf643f45d91945'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:38.740618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd39a103c444535b0bf643f45d91945'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:38.741289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd39a103c444535b0bf643f45d91945'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:38.747184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1355/2710] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:15:38 2022] DELETE /identity/v3/users/2bd39a103c444535b0bf643f45d91945 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:38.751704 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e2c73-80bb-4b99-b82a-85c69b451bd0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:38.752216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e2c73-80bb-4b99-b82a-85c69b451bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:38.752329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e2c73-80bb-4b99-b82a-85c69b451bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:38.779053 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-879e2c73-80bb-4b99-b82a-85c69b451bd0 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:38.780819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-879e2c73-80bb-4b99-b82a-85c69b451bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:38.846461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1356/2711] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:38.856979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:38.858468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:38.944558 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7befe7-4d1e-4d38-bcbb-0e8ebb2a4210 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:38.948348 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7042eb4-3e0b-408a-bdee-e33d6f9e9b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:38.948996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042eb4-3e0b-408a-bdee-e33d6f9e9b31 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:38.949337 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042eb4-3e0b-408a-bdee-e33d6f9e9b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:38.949605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042eb4-3e0b-408a-bdee-e33d6f9e9b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:38.959809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7042eb4-3e0b-408a-bdee-e33d6f9e9b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:38.961306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7042eb4-3e0b-408a-bdee-e33d6f9e9b31 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:38.970959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1356/2712] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:15:39.156454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e2c73-80bb-4b99-b82a-85c69b451bd0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:39.158017 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-879e2c73-80bb-4b99-b82a-85c69b451bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:39.171608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e2c73-80bb-4b99-b82a-85c69b451bd0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:39.176554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:39.177480 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.177758 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.178036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] PATH_INFO: `/v3/projects/71b918ee36fa4888a6d708cfac6c2995` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.179319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b918ee36fa4888a6d708cfac6c2995)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:39.181288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:39.233819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:39.237327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b918ee36fa4888a6d708cfac6c2995', 'request_id': 'req-ab01280d-fed4-4940-92a5-157c1645a824'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:39.246792 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b918ee36fa4888a6d708cfac6c2995', 'request_id': 'req-ab01280d-fed4-4940-92a5-157c1645a824'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:39.255067 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b918ee36fa4888a6d708cfac6c2995', 'request_id': 'req-ab01280d-fed4-4940-92a5-157c1645a824'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:39.267018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] The token cache is being invalidate because project 71b918ee36fa4888a6d708cfac6c2995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:15:39.267309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:39.267991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ab01280d-fed4-4940-92a5-157c1645a824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:15:39.269620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1357/2713] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:15:39 2022] DELETE /identity/v3/projects/71b918ee36fa4888a6d708cfac6c2995 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:39.274817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6be1f3-b023-4e63-92c3-d6076d1aae0b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.275106 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6be1f3-b023-4e63-92c3-d6076d1aae0b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.275393 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6be1f3-b023-4e63-92c3-d6076d1aae0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.330497 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-dd6be1f3-b023-4e63-92c3-d6076d1aae0b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:39.331746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-dd6be1f3-b023-4e63-92c3-d6076d1aae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:39.396541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1357/2714] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:39.405072 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322989fc-4add-48e5-9b1f-9112d2d29cbc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.405377 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322989fc-4add-48e5-9b1f-9112d2d29cbc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.405661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322989fc-4add-48e5-9b1f-9112d2d29cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.426012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-322989fc-4add-48e5-9b1f-9112d2d29cbc None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:39.427259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-322989fc-4add-48e5-9b1f-9112d2d29cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:39.436279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1358/2715] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:39.440621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6be1f3-b023-4e63-92c3-d6076d1aae0b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:39.451594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6be1f3-b023-4e63-92c3-d6076d1aae0b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:39.456344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e712b6-69e7-4af4-a4ba-d65c998a8c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:39.457054 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e712b6-69e7-4af4-a4ba-d65c998a8c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.457368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e712b6-69e7-4af4-a4ba-d65c998a8c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.457642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e712b6-69e7-4af4-a4ba-d65c998a8c4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.458386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e712b6-69e7-4af4-a4ba-d65c998a8c4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:15:39.458692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e712b6-69e7-4af4-a4ba-d65c998a8c4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:39.460317 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e712b6-69e7-4af4-a4ba-d65c998a8c4a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:39.473694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1358/2716] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:15:39.478357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322989fc-4add-48e5-9b1f-9112d2d29cbc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:39.487972 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322989fc-4add-48e5-9b1f-9112d2d29cbc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:39.492730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e927aed-4931-47a4-90ce-2c9b0eb2654c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:39.493437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e927aed-4931-47a4-90ce-2c9b0eb2654c admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.493722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e927aed-4931-47a4-90ce-2c9b0eb2654c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.493998 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e927aed-4931-47a4-90ce-2c9b0eb2654c admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.494694 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e927aed-4931-47a4-90ce-2c9b0eb2654c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:39.496600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e927aed-4931-47a4-90ce-2c9b0eb2654c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:39.523281 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1359/2717] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:39 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:39.527450 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e712b6-69e7-4af4-a4ba-d65c998a8c4a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:39.538147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e712b6-69e7-4af4-a4ba-d65c998a8c4a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:39.556612 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:39.595870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f82d-bf97-406e-9811-073a9e0fe1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:39.596554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f82d-bf97-406e-9811-073a9e0fe1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.596820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f82d-bf97-406e-9811-073a9e0fe1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.597114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f82d-bf97-406e-9811-073a9e0fe1a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.597809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907f82d-bf97-406e-9811-073a9e0fe1a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:39.599714 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907f82d-bf97-406e-9811-073a9e0fe1a3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:39.619630 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1359/2718] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:39 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:15:39.624338 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e927aed-4931-47a4-90ce-2c9b0eb2654c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:39.634775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e927aed-4931-47a4-90ce-2c9b0eb2654c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:39.640048 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547827d8-9d78-4292-824d-807a937ab864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:39.640702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547827d8-9d78-4292-824d-807a937ab864 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.640990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547827d8-9d78-4292-824d-807a937ab864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.641470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547827d8-9d78-4292-824d-807a937ab864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.642210 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547827d8-9d78-4292-824d-807a937ab864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:39.643793 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547827d8-9d78-4292-824d-807a937ab864 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:39.655158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1360/2719] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:39.658979 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f82d-bf97-406e-9811-073a9e0fe1a3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:39.670176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f82d-bf97-406e-9811-073a9e0fe1a3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:39.676153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7cfe6-d148-48af-adf4-8061b14c9fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:39.676817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7cfe6-d148-48af-adf4-8061b14c9fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.677116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7cfe6-d148-48af-adf4-8061b14c9fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.677403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7cfe6-d148-48af-adf4-8061b14c9fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.678130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7cfe6-d148-48af-adf4-8061b14c9fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:39.679712 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7cfe6-d148-48af-adf4-8061b14c9fc1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:39.690384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1360/2720] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:39.695015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547827d8-9d78-4292-824d-807a937ab864 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:39.705746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547827d8-9d78-4292-824d-807a937ab864 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:39.710423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aa492-acab-4bfa-9722-5efddd16ca16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:39.711291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aa492-acab-4bfa-9722-5efddd16ca16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.711579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aa492-acab-4bfa-9722-5efddd16ca16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.711859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aa492-acab-4bfa-9722-5efddd16ca16 admin admin] PATH_INFO: `/v3/projects/193a4ca8d19f449588f3b46389c14f06/users/e334063c001a4435940d7cf850ec350f/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.728308 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aa492-acab-4bfa-9722-5efddd16ca16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=193a4ca8d19f449588f3b46389c14f06, user_id=e334063c001a4435940d7cf850ec350f, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:39.730995 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aa492-acab-4bfa-9722-5efddd16ca16 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:39.743404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1361/2721] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:39 2022] PUT /identity/v3/projects/193a4ca8d19f449588f3b46389c14f06/users/e334063c001a4435940d7cf850ec350f/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:39.749303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff00f2-75ff-4301-8cd9-03b2742a40ea None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.749673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff00f2-75ff-4301-8cd9-03b2742a40ea None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.749955 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff00f2-75ff-4301-8cd9-03b2742a40ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.768409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-44ff00f2-75ff-4301-8cd9-03b2742a40ea None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:39.769709 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-44ff00f2-75ff-4301-8cd9-03b2742a40ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:39.828457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1361/2722] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:39.838052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aa492-acab-4bfa-9722-5efddd16ca16 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:39.839820 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5f0aa492-acab-4bfa-9722-5efddd16ca16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:39.896532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aa492-acab-4bfa-9722-5efddd16ca16 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:39.900136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25c287-c7ab-4a1b-93a9-7505248caba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:39.900770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25c287-c7ab-4a1b-93a9-7505248caba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:39.901057 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25c287-c7ab-4a1b-93a9-7505248caba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:39.901379 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25c287-c7ab-4a1b-93a9-7505248caba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:39.910492 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25c287-c7ab-4a1b-93a9-7505248caba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:39.911927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25c287-c7ab-4a1b-93a9-7505248caba9 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:39.922031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1362/2723] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:44.690230 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff9fc5-4d02-4e66-8e2a-c70a746a79c8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:44.691637 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff9fc5-4d02-4e66-8e2a-c70a746a79c8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:44.692417 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff9fc5-4d02-4e66-8e2a-c70a746a79c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:44.745526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-89ff9fc5-4d02-4e66-8e2a-c70a746a79c8 None None] MFA Rules not processed for user `e334063c001a4435940d7cf850ec350f`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:44.746798 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-89ff9fc5-4d02-4e66-8e2a-c70a746a79c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:44.880089 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1362/2724] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:44.884826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25c287-c7ab-4a1b-93a9-7505248caba9 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:44.900087 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25c287-c7ab-4a1b-93a9-7505248caba9 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:44.911346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a51c24-10ec-48ed-8d87-f9a466d67e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:44.912028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a51c24-10ec-48ed-8d87-f9a466d67e34 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:44.912337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a51c24-10ec-48ed-8d87-f9a466d67e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:44.912599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a51c24-10ec-48ed-8d87-f9a466d67e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:44.913388 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a51c24-10ec-48ed-8d87-f9a466d67e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:44.915328 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a51c24-10ec-48ed-8d87-f9a466d67e34 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:44.954524 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1363/2725] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:15:44 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:44.959051 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff9fc5-4d02-4e66-8e2a-c70a746a79c8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:44.975474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff9fc5-4d02-4e66-8e2a-c70a746a79c8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:45.083536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:45.134999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742af693-3125-4b6a-84be-dfa25e72ad5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:45.135704 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742af693-3125-4b6a-84be-dfa25e72ad5f admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:45.135988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742af693-3125-4b6a-84be-dfa25e72ad5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:45.136269 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742af693-3125-4b6a-84be-dfa25e72ad5f admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:45.136984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742af693-3125-4b6a-84be-dfa25e72ad5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:45.139815 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742af693-3125-4b6a-84be-dfa25e72ad5f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:45.168694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1363/2726] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:15:44 2022] POST /identity/v3/users => generated 298 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:15:45.174798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a51c24-10ec-48ed-8d87-f9a466d67e34 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:45.195364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a51c24-10ec-48ed-8d87-f9a466d67e34 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:45.203416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d310e6-eefa-4c7c-81f1-dd5f10b9fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:45.204676 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d310e6-eefa-4c7c-81f1-dd5f10b9fc3b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:45.205292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d310e6-eefa-4c7c-81f1-dd5f10b9fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:45.205884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d310e6-eefa-4c7c-81f1-dd5f10b9fc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:45.207354 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d310e6-eefa-4c7c-81f1-dd5f10b9fc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:45.210685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d310e6-eefa-4c7c-81f1-dd5f10b9fc3b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:45.243222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1364/2727] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:45.245115 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742af693-3125-4b6a-84be-dfa25e72ad5f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:45.260423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742af693-3125-4b6a-84be-dfa25e72ad5f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:45.268837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2e4b-8c58-43d6-9011-8d342da86aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:45.270960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2e4b-8c58-43d6-9011-8d342da86aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:45.271845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2e4b-8c58-43d6-9011-8d342da86aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:45.272644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2e4b-8c58-43d6-9011-8d342da86aa3 admin admin] PATH_INFO: `/v3/projects/0f66b168088f4a3d8098216c5ddd30aa/users/4dd7d208141c47cebca06e278f827d21/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:45.312676 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c2e4b-8c58-43d6-9011-8d342da86aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f66b168088f4a3d8098216c5ddd30aa, user_id=4dd7d208141c47cebca06e278f827d21, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:45.316526 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c2e4b-8c58-43d6-9011-8d342da86aa3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:45.354707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1364/2728] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:45 2022] PUT /identity/v3/projects/0f66b168088f4a3d8098216c5ddd30aa/users/4dd7d208141c47cebca06e278f827d21/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:45.356110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d310e6-eefa-4c7c-81f1-dd5f10b9fc3b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:45.374889 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d310e6-eefa-4c7c-81f1-dd5f10b9fc3b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:45.438819 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:45.468231 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daf135-fbb9-4c34-a61a-91b96c9be7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:45.468826 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daf135-fbb9-4c34-a61a-91b96c9be7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:45.469140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daf135-fbb9-4c34-a61a-91b96c9be7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:45.469430 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daf135-fbb9-4c34-a61a-91b96c9be7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:45.470165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1daf135-fbb9-4c34-a61a-91b96c9be7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:45.471782 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1daf135-fbb9-4c34-a61a-91b96c9be7c2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:45.484828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1365/2729] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:15:45.489045 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2e4b-8c58-43d6-9011-8d342da86aa3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:45.504691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2e4b-8c58-43d6-9011-8d342da86aa3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:45.514167 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6edd00-f582-4256-a68b-ff72cec6b6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:45.515083 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6edd00-f582-4256-a68b-ff72cec6b6ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:45.515367 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6edd00-f582-4256-a68b-ff72cec6b6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:45.515655 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6edd00-f582-4256-a68b-ff72cec6b6ca admin admin] PATH_INFO: `/v3/projects/0f66b168088f4a3d8098216c5ddd30aa/users/4dd7d208141c47cebca06e278f827d21/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:45.519020 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6edd00-f582-4256-a68b-ff72cec6b6ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f66b168088f4a3d8098216c5ddd30aa, user_id=4dd7d208141c47cebca06e278f827d21, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:45.521774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6edd00-f582-4256-a68b-ff72cec6b6ca admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:45.535934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1365/2730] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:15:45 2022] PUT /identity/v3/projects/0f66b168088f4a3d8098216c5ddd30aa/users/4dd7d208141c47cebca06e278f827d21/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:15:50.621917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d13fb-9ece-4dd1-a678-ba2f5468e911 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:50.621917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d13fb-9ece-4dd1-a678-ba2f5468e911 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:50.621917 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d13fb-9ece-4dd1-a678-ba2f5468e911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:50.658324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8a4d13fb-9ece-4dd1-a678-ba2f5468e911 None None] MFA Rules not processed for user `4dd7d208141c47cebca06e278f827d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:50.659726 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8a4d13fb-9ece-4dd1-a678-ba2f5468e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:50.780704 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1366/2731] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:50.793372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6edd00-f582-4256-a68b-ff72cec6b6ca admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:50.795432 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2d6edd00-f582-4256-a68b-ff72cec6b6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:50.864985 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6edd00-f582-4256-a68b-ff72cec6b6ca admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:50.869511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64384f-ef62-4cb2-9444-d58571f67c13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915704f26c814eaa9e6c2a68533a9c66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:50.870191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64384f-ef62-4cb2-9444-d58571f67c13 service glance] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:50.870845 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64384f-ef62-4cb2-9444-d58571f67c13 service glance] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:50.871241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64384f-ef62-4cb2-9444-d58571f67c13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:50.989876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d13fb-9ece-4dd1-a678-ba2f5468e911 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:50.991640 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8a4d13fb-9ece-4dd1-a678-ba2f5468e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:50.992594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64384f-ef62-4cb2-9444-d58571f67c13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:50.994598 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64384f-ef62-4cb2-9444-d58571f67c13 service glance] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:51.011356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1366/2732] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:51.167159 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d13fb-9ece-4dd1-a678-ba2f5468e911 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:51.180389 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:51.181070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:51.182062 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:51.183569 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:51.189532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:51.202705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acd254-4d46-494b-8ae4-12a8158ef84c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:51.202998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acd254-4d46-494b-8ae4-12a8158ef84c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:51.203284 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acd254-4d46-494b-8ae4-12a8158ef84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:51.215956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:51.218154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:51.233611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-91acd254-4d46-494b-8ae4-12a8158ef84c None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:51.239160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1367/2733] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:51.248675 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-91acd254-4d46-494b-8ae4-12a8158ef84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:51.331336 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:51.398305 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1367/2734] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:15:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:15:51.414035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:51.416176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:51.485342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:15:51.542265 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6d7b3-537e-4741-a783-345f21f0c4a7 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:51.548560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfeca1-31ac-4f90-8ec7-7423423466ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:51.549337 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfeca1-31ac-4f90-8ec7-7423423466ab service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:51.549732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfeca1-31ac-4f90-8ec7-7423423466ab service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:51.550215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfeca1-31ac-4f90-8ec7-7423423466ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:51.560170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfeca1-31ac-4f90-8ec7-7423423466ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:51.561890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfeca1-31ac-4f90-8ec7-7423423466ab service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:51.571532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1368/2735] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:15:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:15:55.598736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e5d15-ea72-4424-b82e-cb37ac3df507 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:55.600324 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e5d15-ea72-4424-b82e-cb37ac3df507 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:55.601469 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e5d15-ea72-4424-b82e-cb37ac3df507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:55.633589 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2d1e5d15-ea72-4424-b82e-cb37ac3df507 None None] MFA Rules not processed for user `25945373ee26489e8014f07ec27d81f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:15:55.635242 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2d1e5d15-ea72-4424-b82e-cb37ac3df507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:55.738127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1368/2736] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:15:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:15:56.225997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfeca1-31ac-4f90-8ec7-7423423466ab service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:56.307602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfeca1-31ac-4f90-8ec7-7423423466ab service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:56.318123 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467f460-b278-4c50-a0c5-eac4bab7988a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:56.319498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f460-b278-4c50-a0c5-eac4bab7988a service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:56.319986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f460-b278-4c50-a0c5-eac4bab7988a service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:56.320461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f460-b278-4c50-a0c5-eac4bab7988a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:56.334393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d467f460-b278-4c50-a0c5-eac4bab7988a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:56.351255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467f460-b278-4c50-a0c5-eac4bab7988a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:56.353187 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467f460-b278-4c50-a0c5-eac4bab7988a service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:56.363451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1369/2737] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:15:59.095415 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e5d15-ea72-4424-b82e-cb37ac3df507 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:15:59.097040 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2d1e5d15-ea72-4424-b82e-cb37ac3df507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:59.241840 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e5d15-ea72-4424-b82e-cb37ac3df507 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:15:59.251806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095d3e2-f8ac-43b0-9933-ac6b5a6a146a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:15:59.252481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095d3e2-f8ac-43b0-9933-ac6b5a6a146a service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:15:59.252774 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095d3e2-f8ac-43b0-9933-ac6b5a6a146a service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:15:59.253057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095d3e2-f8ac-43b0-9933-ac6b5a6a146a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:15:59.256257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1095d3e2-f8ac-43b0-9933-ac6b5a6a146a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:15:59.270789 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095d3e2-f8ac-43b0-9933-ac6b5a6a146a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:15:59.272905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095d3e2-f8ac-43b0-9933-ac6b5a6a146a service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:15:59.304144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1369/2738] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:15:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:16:02.088681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7ac4-569e-4006-9196-54acce113718 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:16:02.089244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7ac4-569e-4006-9196-54acce113718 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:16:02.089493 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7ac4-569e-4006-9196-54acce113718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:16:02.106353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8bfc7ac4-569e-4006-9196-54acce113718 None None] MFA Rules not processed for user `e334063c001a4435940d7cf850ec350f`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:16:02.107542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8bfc7ac4-569e-4006-9196-54acce113718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:16:02.114832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1370/2739] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:16:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:16:03.965320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a30592-e403-4fb1-b499-4b8ef5e3bcd0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:16:03.965920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a30592-e403-4fb1-b499-4b8ef5e3bcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:16:03.966186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a30592-e403-4fb1-b499-4b8ef5e3bcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:16:03.992757 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f3a30592-e403-4fb1-b499-4b8ef5e3bcd0 None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:16:03.994038 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f3a30592-e403-4fb1-b499-4b8ef5e3bcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:16:04.002228 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1370/2740] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jun 21 19:16:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:16:04.014661 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc7ac4-569e-4006-9196-54acce113718 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:16:04.026743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc7ac4-569e-4006-9196-54acce113718 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:16:04.031818 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f07148-5b47-4932-a103-7dee29a1040d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:16:04.032462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f07148-5b47-4932-a103-7dee29a1040d service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:16:04.032746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f07148-5b47-4932-a103-7dee29a1040d service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:16:04.033061 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f07148-5b47-4932-a103-7dee29a1040d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:16:04.041403 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f07148-5b47-4932-a103-7dee29a1040d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:16:04.042991 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f07148-5b47-4932-a103-7dee29a1040d service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:16:04.051943 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1371/2741] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:16:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:16:13.987849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=82 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:16:14.048593 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=53 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:16:36.160095 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a30592-e403-4fb1-b499-4b8ef5e3bcd0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:16:36.171302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a30592-e403-4fb1-b499-4b8ef5e3bcd0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:16:36.175356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e7ca4-d8ef-4224-90f1-0145cedecc06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:16:36.176130 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e7ca4-d8ef-4224-90f1-0145cedecc06 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:16:36.176477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e7ca4-d8ef-4224-90f1-0145cedecc06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:16:36.176763 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e7ca4-d8ef-4224-90f1-0145cedecc06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:16:36.188236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e7ca4-d8ef-4224-90f1-0145cedecc06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:16:36.190481 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e7ca4-d8ef-4224-90f1-0145cedecc06 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:16:36.209156 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1371/2742] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:16:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:16:46.200084 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:17:47.383256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f07148-5b47-4932-a103-7dee29a1040d service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:17:47.385742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-24f07148-5b47-4932-a103-7dee29a1040d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:17:47.480437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f07148-5b47-4932-a103-7dee29a1040d service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:17:47.484426 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e12089-0cd8-41b1-9b9b-665074f5d10a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:17:47.485110 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e12089-0cd8-41b1-9b9b-665074f5d10a service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:17:47.485407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e12089-0cd8-41b1-9b9b-665074f5d10a service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:17:47.485683 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e12089-0cd8-41b1-9b9b-665074f5d10a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:17:47.488711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f8e12089-0cd8-41b1-9b9b-665074f5d10a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:17:47.504462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e12089-0cd8-41b1-9b9b-665074f5d10a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:17:47.505940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e12089-0cd8-41b1-9b9b-665074f5d10a service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:17:47.519129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1372/2743] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:17:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:17:53.668868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e7ca4-d8ef-4224-90f1-0145cedecc06 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:17:53.678796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e7ca4-d8ef-4224-90f1-0145cedecc06 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:17:53.683451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d96580-9c3a-451b-bd22-ae5bcde41b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:17:53.684194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d96580-9c3a-451b-bd22-ae5bcde41b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:17:53.684560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d96580-9c3a-451b-bd22-ae5bcde41b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:17:53.684887 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d96580-9c3a-451b-bd22-ae5bcde41b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:17:53.687421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-70d96580-9c3a-451b-bd22-ae5bcde41b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:17:53.703100 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d96580-9c3a-451b-bd22-ae5bcde41b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:17:53.704586 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d96580-9c3a-451b-bd22-ae5bcde41b85 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:17:53.718306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1372/2744] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:17:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:17:57.513723 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=35 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:18:03.711654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:18:04.809953 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d367064-0f02-4ee7-adeb-c60db554897d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:18:04.810630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d367064-0f02-4ee7-adeb-c60db554897d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:18:04.810630 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d367064-0f02-4ee7-adeb-c60db554897d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:18:04.830015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7d367064-0f02-4ee7-adeb-c60db554897d None None] MFA Rules not processed for user `25945373ee26489e8014f07ec27d81f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:18:04.831450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7d367064-0f02-4ee7-adeb-c60db554897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:18:04.839813 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1373/2745] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:18:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:18:05.309755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d96580-9c3a-451b-bd22-ae5bcde41b85 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:18:05.320501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d96580-9c3a-451b-bd22-ae5bcde41b85 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:18:05.326047 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:18:05.326922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:18:05.327214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:18:05.327493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:18:05.330542 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:18:05.345214 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:18:05.346734 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:18:05.357042 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1373/2746] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:18:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:18:14.824450 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:18:15.351744 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:18:19.545394 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d367064-0f02-4ee7-adeb-c60db554897d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:18:19.557330 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d367064-0f02-4ee7-adeb-c60db554897d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:18:19.563078 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e1fbd-bec7-4481-9054-9f393fa54770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:18:19.564205 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e1fbd-bec7-4481-9054-9f393fa54770 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:18:19.564665 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e1fbd-bec7-4481-9054-9f393fa54770 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:18:19.565289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e1fbd-bec7-4481-9054-9f393fa54770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:18:19.568773 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ad1e1fbd-bec7-4481-9054-9f393fa54770 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:18:19.589183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e1fbd-bec7-4481-9054-9f393fa54770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:18:19.591977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e1fbd-bec7-4481-9054-9f393fa54770 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:18:19.607166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1374/2747] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:18:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:18:29.600951 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:19:09.553306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:09.555441 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:09.579031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0870e78-9956-47c2-9943-2c65fdef6c52 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:09.585226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19467a4-3eef-4068-80a4-ba1d9b714cd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:09.585894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19467a4-3eef-4068-80a4-ba1d9b714cd4 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:09.586188 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19467a4-3eef-4068-80a4-ba1d9b714cd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:09.586466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19467a4-3eef-4068-80a4-ba1d9b714cd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:09.588409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c19467a4-3eef-4068-80a4-ba1d9b714cd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:09.607114 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19467a4-3eef-4068-80a4-ba1d9b714cd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:09.609653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19467a4-3eef-4068-80a4-ba1d9b714cd4 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:09.627560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1374/2748] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:19:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:19:10.748257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e1fbd-bec7-4481-9054-9f393fa54770 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:10.764910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e1fbd-bec7-4481-9054-9f393fa54770 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:10.771821 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd28e3-21cc-44ff-a523-fcd3d1af7856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcbcec65220c4ebb95f974925fa73f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:10.772565 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd28e3-21cc-44ff-a523-fcd3d1af7856 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:10.772853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd28e3-21cc-44ff-a523-fcd3d1af7856 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:10.773163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd28e3-21cc-44ff-a523-fcd3d1af7856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:10.775740 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0cbd28e3-21cc-44ff-a523-fcd3d1af7856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:10.793698 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd28e3-21cc-44ff-a523-fcd3d1af7856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:10.795529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd28e3-21cc-44ff-a523-fcd3d1af7856 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:10.809570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1375/2749] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:19:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:19:17.118408 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11fdef-bb0f-4706-902c-c83eb902f0a5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:17.119004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11fdef-bb0f-4706-902c-c83eb902f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:17.119110 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11fdef-bb0f-4706-902c-c83eb902f0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:17.180377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8a11fdef-bb0f-4706-902c-c83eb902f0a5 None None] MFA Rules not processed for user `9fee9b6e04f24f96adb1a70306463a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:19:17.181850 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8a11fdef-bb0f-4706-902c-c83eb902f0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:17.201286 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1375/2750] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jun 21 19:19:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:19:17.251437 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd28e3-21cc-44ff-a523-fcd3d1af7856 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:17.281848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd28e3-21cc-44ff-a523-fcd3d1af7856 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:17.287118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce269de-f172-4ac3-a4fb-9d9ff74af5ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad2a56d3235b46e3b1fbb8bf0c2b4e0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:17.287785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce269de-f172-4ac3-a4fb-9d9ff74af5ef service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:17.288069 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce269de-f172-4ac3-a4fb-9d9ff74af5ef service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:17.288346 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce269de-f172-4ac3-a4fb-9d9ff74af5ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:17.297986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce269de-f172-4ac3-a4fb-9d9ff74af5ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:17.299591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce269de-f172-4ac3-a4fb-9d9ff74af5ef service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:17.308463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1376/2751] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jun 21 19:19:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 19:19:20.880196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11fdef-bb0f-4706-902c-c83eb902f0a5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:20.882175 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8a11fdef-bb0f-4706-902c-c83eb902f0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:20.897974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11fdef-bb0f-4706-902c-c83eb902f0a5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:20.903016 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2cce6-f8b2-4730-89e3-8ac789a922d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:20.903669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2cce6-f8b2-4730-89e3-8ac789a922d6 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:20.903951 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2cce6-f8b2-4730-89e3-8ac789a922d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:20.904232 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2cce6-f8b2-4730-89e3-8ac789a922d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:20.905999 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-28a2cce6-f8b2-4730-89e3-8ac789a922d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:20.918329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2cce6-f8b2-4730-89e3-8ac789a922d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:20.919834 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2cce6-f8b2-4730-89e3-8ac789a922d6 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:20.934477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1376/2752] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:19:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:19:23.888227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce269de-f172-4ac3-a4fb-9d9ff74af5ef service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:23.900160 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce269de-f172-4ac3-a4fb-9d9ff74af5ef service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:23.905719 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95243f77-0a71-4d3e-902e-f7937cc4068a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:23.906452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95243f77-0a71-4d3e-902e-f7937cc4068a service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:23.906806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95243f77-0a71-4d3e-902e-f7937cc4068a service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:23.907154 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95243f77-0a71-4d3e-902e-f7937cc4068a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:23.909044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-95243f77-0a71-4d3e-902e-f7937cc4068a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:23.923680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95243f77-0a71-4d3e-902e-f7937cc4068a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:23.927075 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95243f77-0a71-4d3e-902e-f7937cc4068a service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:23.946580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1377/2753] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:19:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:19:23.948807 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2cce6-f8b2-4730-89e3-8ac789a922d6 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:24.016755 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2cce6-f8b2-4730-89e3-8ac789a922d6 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:24.021333 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:24.022200 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:24.022488 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:24.022768 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] PATH_INFO: `/v3/users/e334063c001a4435940d7cf850ec350f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:24.023952 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e334063c001a4435940d7cf850ec350f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:24.025967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:24.083624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e334063c001a4435940d7cf850ec350f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.096163 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e334063c001a4435940d7cf850ec350f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.104416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e334063c001a4435940d7cf850ec350f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.112120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e334063c001a4435940d7cf850ec350f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.112761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e334063c001a4435940d7cf850ec350f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.120096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1377/2754] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:19:23 2022] DELETE /identity/v3/users/e334063c001a4435940d7cf850ec350f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:19:24.303056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95243f77-0a71-4d3e-902e-f7937cc4068a service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:24.313799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95243f77-0a71-4d3e-902e-f7937cc4068a service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:24.374246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:24.375108 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:24.375393 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:24.375670 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] PATH_INFO: `/v3/projects/193a4ca8d19f449588f3b46389c14f06` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:24.376849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=193a4ca8d19f449588f3b46389c14f06)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:24.378776 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:24.429364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193a4ca8d19f449588f3b46389c14f06', 'request_id': 'req-53f52654-cc11-44b1-8fe9-18841a5dd3f5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.435808 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193a4ca8d19f449588f3b46389c14f06', 'request_id': 'req-53f52654-cc11-44b1-8fe9-18841a5dd3f5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.442862 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193a4ca8d19f449588f3b46389c14f06', 'request_id': 'req-53f52654-cc11-44b1-8fe9-18841a5dd3f5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.454654 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] The token cache is being invalidate because project 193a4ca8d19f449588f3b46389c14f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:19:24.454928 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.455600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.457194 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1378/2755] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:19:24 2022] DELETE /identity/v3/projects/193a4ca8d19f449588f3b46389c14f06 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:19:24.461282 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:24.463270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:24.472858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:19:24.477113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463438b-89c0-4b5d-bc6e-f5d580349595 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:24.537289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:24.538161 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:24.538448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:24.538725 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] PATH_INFO: `/v3/users/4dd7d208141c47cebca06e278f827d21` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:24.539905 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd7d208141c47cebca06e278f827d21)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:24.541891 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:24.588922 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd7d208141c47cebca06e278f827d21'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.600866 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd7d208141c47cebca06e278f827d21'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.612281 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd7d208141c47cebca06e278f827d21'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.620963 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd7d208141c47cebca06e278f827d21'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.621764 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd7d208141c47cebca06e278f827d21'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.628257 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1378/2756] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:19:24 2022] DELETE /identity/v3/users/4dd7d208141c47cebca06e278f827d21 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:19:24.806271 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:24.807779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:24.823590 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52654-cc11-44b1-8fe9-18841a5dd3f5 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:24.866401 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:19:24.886058 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:24.886927 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:24.887212 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:24.887517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] PATH_INFO: `/v3/projects/0f66b168088f4a3d8098216c5ddd30aa` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:24.888711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f66b168088f4a3d8098216c5ddd30aa)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:24.890700 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:24.937546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f66b168088f4a3d8098216c5ddd30aa', 'request_id': 'req-62227603-aea4-49c7-9c45-76fbaa6ed8de'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.943796 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f66b168088f4a3d8098216c5ddd30aa', 'request_id': 'req-62227603-aea4-49c7-9c45-76fbaa6ed8de'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.952729 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f66b168088f4a3d8098216c5ddd30aa', 'request_id': 'req-62227603-aea4-49c7-9c45-76fbaa6ed8de'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.963664 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] The token cache is being invalidate because project 0f66b168088f4a3d8098216c5ddd30aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:19:24.964279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.964614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:19:24.966209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1379/2757] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:19:24 2022] DELETE /identity/v3/projects/0f66b168088f4a3d8098216c5ddd30aa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:19:34.624343 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:19:34.960243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:19:58.551260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:19:58.552766 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:58.621562 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8655594-d8c2-4c28-af31-71814b334104 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:19:58.679782 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027d937-9d6c-4668-a6e1-fd7c1a5847b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:19:58.680461 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027d937-9d6c-4668-a6e1-fd7c1a5847b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:19:58.680746 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027d937-9d6c-4668-a6e1-fd7c1a5847b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:19:58.681029 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027d937-9d6c-4668-a6e1-fd7c1a5847b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:19:58.684895 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6027d937-9d6c-4668-a6e1-fd7c1a5847b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:19:58.758595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027d937-9d6c-4668-a6e1-fd7c1a5847b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:19:58.760152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027d937-9d6c-4668-a6e1-fd7c1a5847b4 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:19:58.771144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1379/2758] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:20:02.343107 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:02.345232 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:02.365542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62227603-aea4-49c7-9c45-76fbaa6ed8de admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:02.371239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:02.372148 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:02.372428 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:02.372707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] PATH_INFO: `/v3/users/25945373ee26489e8014f07ec27d81f7` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:02.374043 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25945373ee26489e8014f07ec27d81f7)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:02.375974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:02.443279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25945373ee26489e8014f07ec27d81f7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.457245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25945373ee26489e8014f07ec27d81f7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.468262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25945373ee26489e8014f07ec27d81f7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.469554 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25945373ee26489e8014f07ec27d81f7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.480395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-08df7639-dc36-4fe5-a451-5bb8f4c095f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25945373ee26489e8014f07ec27d81f7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.490264 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1380/2759] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:02 2022] DELETE /identity/v3/users/25945373ee26489e8014f07ec27d81f7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:02.698623 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027d937-9d6c-4668-a6e1-fd7c1a5847b4 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:02.712291 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027d937-9d6c-4668-a6e1-fd7c1a5847b4 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:02.799197 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:02.799996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:02.800302 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:02.800823 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] PATH_INFO: `/v3/projects/d197f69e1d4b49d0801e6b625599aad8` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:02.802207 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d197f69e1d4b49d0801e6b625599aad8)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:02.804368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:02.814076 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:02.859436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd197f69e1d4b49d0801e6b625599aad8', 'request_id': 'req-9bc9f3d3-0641-4201-aa22-8959bea3f401'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.866529 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd197f69e1d4b49d0801e6b625599aad8', 'request_id': 'req-9bc9f3d3-0641-4201-aa22-8959bea3f401'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.874609 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd197f69e1d4b49d0801e6b625599aad8', 'request_id': 'req-9bc9f3d3-0641-4201-aa22-8959bea3f401'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.888595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] The token cache is being invalidate because project d197f69e1d4b49d0801e6b625599aad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:02.888883 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.889574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9bc9f3d3-0641-4201-aa22-8959bea3f401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:02.891145 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1380/2760] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:02 2022] DELETE /identity/v3/projects/d197f69e1d4b49d0801e6b625599aad8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:02.896620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98251835-bf4c-4b4e-89eb-f58161b9144b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:02.896978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98251835-bf4c-4b4e-89eb-f58161b9144b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:02.897340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98251835-bf4c-4b4e-89eb-f58161b9144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:02.917291 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-98251835-bf4c-4b4e-89eb-f58161b9144b None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:02.918525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-98251835-bf4c-4b4e-89eb-f58161b9144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:03.020629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1381/2761] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:03.029814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e9a8-d947-4452-8bc3-116e00d9e947 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.030167 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e9a8-d947-4452-8bc3-116e00d9e947 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.030498 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e9a8-d947-4452-8bc3-116e00d9e947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.059385 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9773e9a8-d947-4452-8bc3-116e00d9e947 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:03.061216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9773e9a8-d947-4452-8bc3-116e00d9e947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:03.075854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1381/2762] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:03.081519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98251835-bf4c-4b4e-89eb-f58161b9144b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.096247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98251835-bf4c-4b4e-89eb-f58161b9144b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.101897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7518f30-5548-4208-a156-41114ecb4ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.102606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7518f30-5548-4208-a156-41114ecb4ace admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.102892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7518f30-5548-4208-a156-41114ecb4ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.103168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7518f30-5548-4208-a156-41114ecb4ace admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.103905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7518f30-5548-4208-a156-41114ecb4ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:20:03.104214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7518f30-5548-4208-a156-41114ecb4ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.105865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7518f30-5548-4208-a156-41114ecb4ace admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.118605 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1382/2763] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:20:03.124899 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e9a8-d947-4452-8bc3-116e00d9e947 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.135857 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e9a8-d947-4452-8bc3-116e00d9e947 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.140672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9391bb-1e6d-40cc-aacf-fb18cd0560cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.141389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9391bb-1e6d-40cc-aacf-fb18cd0560cc admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.141827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9391bb-1e6d-40cc-aacf-fb18cd0560cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.142153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9391bb-1e6d-40cc-aacf-fb18cd0560cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.142906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9391bb-1e6d-40cc-aacf-fb18cd0560cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.145171 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9391bb-1e6d-40cc-aacf-fb18cd0560cc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.172920 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1382/2764] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:03 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:20:03.177222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7518f30-5548-4208-a156-41114ecb4ace admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.188332 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7518f30-5548-4208-a156-41114ecb4ace admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.252244 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d7d15-6dd4-4b38-a2d8-137809fd382f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.252926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d7d15-6dd4-4b38-a2d8-137809fd382f admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.253252 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d7d15-6dd4-4b38-a2d8-137809fd382f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.253537 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d7d15-6dd4-4b38-a2d8-137809fd382f admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.254243 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d7d15-6dd4-4b38-a2d8-137809fd382f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.256191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d7d15-6dd4-4b38-a2d8-137809fd382f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.264030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:03.278730 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1383/2765] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:03 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:03.283431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9391bb-1e6d-40cc-aacf-fb18cd0560cc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.294339 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9391bb-1e6d-40cc-aacf-fb18cd0560cc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.299167 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49da88a-80d2-4528-988c-7df111fef7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.299844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49da88a-80d2-4528-988c-7df111fef7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.300136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49da88a-80d2-4528-988c-7df111fef7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.300422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49da88a-80d2-4528-988c-7df111fef7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.301190 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49da88a-80d2-4528-988c-7df111fef7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.302827 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49da88a-80d2-4528-988c-7df111fef7b5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.315196 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1383/2766] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:03.320012 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d7d15-6dd4-4b38-a2d8-137809fd382f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.334910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d7d15-6dd4-4b38-a2d8-137809fd382f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.340082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa6a74-465a-4da0-b334-9cc6f1847c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.340784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa6a74-465a-4da0-b334-9cc6f1847c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.341046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa6a74-465a-4da0-b334-9cc6f1847c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.341353 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa6a74-465a-4da0-b334-9cc6f1847c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.342083 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa6a74-465a-4da0-b334-9cc6f1847c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.343699 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa6a74-465a-4da0-b334-9cc6f1847c60 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.354763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1384/2767] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:03.359806 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49da88a-80d2-4528-988c-7df111fef7b5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.370907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49da88a-80d2-4528-988c-7df111fef7b5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.375742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0292cf-aa76-46d2-95bd-e927fa59993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.376662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0292cf-aa76-46d2-95bd-e927fa59993f admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.376944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0292cf-aa76-46d2-95bd-e927fa59993f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.377252 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0292cf-aa76-46d2-95bd-e927fa59993f admin admin] PATH_INFO: `/v3/projects/e344d445db0c42a3a7f26efba5f734d8/users/2f9013432d3e4ae39e40dbe6d8a26172/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.393717 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0292cf-aa76-46d2-95bd-e927fa59993f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e344d445db0c42a3a7f26efba5f734d8, user_id=2f9013432d3e4ae39e40dbe6d8a26172, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.396384 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0292cf-aa76-46d2-95bd-e927fa59993f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.412074 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1384/2768] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:03 2022] PUT /identity/v3/projects/e344d445db0c42a3a7f26efba5f734d8/users/2f9013432d3e4ae39e40dbe6d8a26172/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:03.423081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a9e9a-2ef8-40d3-ad89-1205e6ca2c3d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.423552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a9e9a-2ef8-40d3-ad89-1205e6ca2c3d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.423860 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a9e9a-2ef8-40d3-ad89-1205e6ca2c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.460046 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f34a9e9a-2ef8-40d3-ad89-1205e6ca2c3d None None] MFA Rules not processed for user `2f9013432d3e4ae39e40dbe6d8a26172`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:03.462247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f34a9e9a-2ef8-40d3-ad89-1205e6ca2c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:03.547446 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1385/2769] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:03.552079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0292cf-aa76-46d2-95bd-e927fa59993f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.626865 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0292cf-aa76-46d2-95bd-e927fa59993f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.632053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac080e8f-52b6-484f-8ac0-6494fc832a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.632852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac080e8f-52b6-484f-8ac0-6494fc832a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.633249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac080e8f-52b6-484f-8ac0-6494fc832a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.633536 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac080e8f-52b6-484f-8ac0-6494fc832a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.634344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac080e8f-52b6-484f-8ac0-6494fc832a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.636914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac080e8f-52b6-484f-8ac0-6494fc832a4d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.667991 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1385/2770] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:03 2022] POST /identity/v3/projects => generated 405 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:03.669987 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a9e9a-2ef8-40d3-ad89-1205e6ca2c3d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.681183 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a9e9a-2ef8-40d3-ad89-1205e6ca2c3d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.740899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2741a-9d6a-4e5a-b1b7-e127c4f34b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.741611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2741a-9d6a-4e5a-b1b7-e127c4f34b34 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.741899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2741a-9d6a-4e5a-b1b7-e127c4f34b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.742189 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2741a-9d6a-4e5a-b1b7-e127c4f34b34 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.742897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2741a-9d6a-4e5a-b1b7-e127c4f34b34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.744857 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2741a-9d6a-4e5a-b1b7-e127c4f34b34 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.766746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1386/2771] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:03 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:03.770906 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac080e8f-52b6-484f-8ac0-6494fc832a4d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.782068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac080e8f-52b6-484f-8ac0-6494fc832a4d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.786769 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fc16b-e3a4-44e6-a61e-0e3539f8e836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.787442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fc16b-e3a4-44e6-a61e-0e3539f8e836 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.787728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fc16b-e3a4-44e6-a61e-0e3539f8e836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.788008 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fc16b-e3a4-44e6-a61e-0e3539f8e836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.788748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fc16b-e3a4-44e6-a61e-0e3539f8e836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.790403 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fc16b-e3a4-44e6-a61e-0e3539f8e836 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.801804 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1386/2772] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:03.806325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2741a-9d6a-4e5a-b1b7-e127c4f34b34 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.816530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2741a-9d6a-4e5a-b1b7-e127c4f34b34 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.821628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e62f95-2355-4f01-b479-3f83684708d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.823221 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62f95-2355-4f01-b479-3f83684708d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.823502 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62f95-2355-4f01-b479-3f83684708d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.823771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62f95-2355-4f01-b479-3f83684708d6 admin admin] PATH_INFO: `/v3/projects/b75a06eacf2f4b1bb2ca4c52dfe56cc1/users/7dbee1bce0e643acbaab424578fb8c32/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.842091 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e62f95-2355-4f01-b479-3f83684708d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75a06eacf2f4b1bb2ca4c52dfe56cc1, user_id=7dbee1bce0e643acbaab424578fb8c32, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.844777 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e62f95-2355-4f01-b479-3f83684708d6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.860028 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1387/2773] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:03 2022] PUT /identity/v3/projects/b75a06eacf2f4b1bb2ca4c52dfe56cc1/users/7dbee1bce0e643acbaab424578fb8c32/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:03.864176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fc16b-e3a4-44e6-a61e-0e3539f8e836 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.875251 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fc16b-e3a4-44e6-a61e-0e3539f8e836 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.879313 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:03.950128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745fd9d-09c6-4114-9d63-c2d20e550c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.950821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745fd9d-09c6-4114-9d63-c2d20e550c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.951094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745fd9d-09c6-4114-9d63-c2d20e550c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.951372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745fd9d-09c6-4114-9d63-c2d20e550c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.952111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745fd9d-09c6-4114-9d63-c2d20e550c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.953796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745fd9d-09c6-4114-9d63-c2d20e550c45 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:03.965448 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1387/2774] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:03.970082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e62f95-2355-4f01-b479-3f83684708d6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:03.981961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e62f95-2355-4f01-b479-3f83684708d6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:03.987064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43643a-ed22-4df4-b74a-35c4998fddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:03.987974 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43643a-ed22-4df4-b74a-35c4998fddf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:03.988259 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43643a-ed22-4df4-b74a-35c4998fddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:03.988532 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43643a-ed22-4df4-b74a-35c4998fddf0 admin admin] PATH_INFO: `/v3/projects/b75a06eacf2f4b1bb2ca4c52dfe56cc1/users/7dbee1bce0e643acbaab424578fb8c32/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:03.990576 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43643a-ed22-4df4-b74a-35c4998fddf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75a06eacf2f4b1bb2ca4c52dfe56cc1, user_id=7dbee1bce0e643acbaab424578fb8c32, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:03.993279 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43643a-ed22-4df4-b74a-35c4998fddf0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:04.004746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1388/2775] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:03 2022] PUT /identity/v3/projects/b75a06eacf2f4b1bb2ca4c52dfe56cc1/users/7dbee1bce0e643acbaab424578fb8c32/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:04.016791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a01bf-0496-4105-8443-801cd84190bd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:04.017128 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a01bf-0496-4105-8443-801cd84190bd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:04.017389 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a01bf-0496-4105-8443-801cd84190bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:04.048945 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c16a01bf-0496-4105-8443-801cd84190bd None None] MFA Rules not processed for user `7dbee1bce0e643acbaab424578fb8c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:04.050227 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c16a01bf-0496-4105-8443-801cd84190bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:04.129552 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1388/2776] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:04 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:04.140903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43643a-ed22-4df4-b74a-35c4998fddf0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:04.142677 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2c43643a-ed22-4df4-b74a-35c4998fddf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:04.167016 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:04.206236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43643a-ed22-4df4-b74a-35c4998fddf0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:04.210093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af5a0c-7498-4446-be68-3e0015d9b383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:04.210774 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af5a0c-7498-4446-be68-3e0015d9b383 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:04.211031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af5a0c-7498-4446-be68-3e0015d9b383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:04.211315 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af5a0c-7498-4446-be68-3e0015d9b383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:04.221976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af5a0c-7498-4446-be68-3e0015d9b383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:04.223416 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af5a0c-7498-4446-be68-3e0015d9b383 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:04.233977 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1389/2777] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:04 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:04.367565 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a01bf-0496-4105-8443-801cd84190bd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:04.430628 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a01bf-0496-4105-8443-801cd84190bd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:04.435194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:04.436078 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:04.436357 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:04.436640 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] PATH_INFO: `/v3/users/2f9013432d3e4ae39e40dbe6d8a26172` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:04.437897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9013432d3e4ae39e40dbe6d8a26172)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:04.439833 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:04.477235 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:04.491511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9013432d3e4ae39e40dbe6d8a26172'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:04.503470 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9013432d3e4ae39e40dbe6d8a26172'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:04.513648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9013432d3e4ae39e40dbe6d8a26172'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:04.522496 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9013432d3e4ae39e40dbe6d8a26172'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:04.523835 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9013432d3e4ae39e40dbe6d8a26172'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:04.531169 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1389/2778] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:04 2022] DELETE /identity/v3/users/2f9013432d3e4ae39e40dbe6d8a26172 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:04.536149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080df7e-9118-4469-bc93-f68d468c3140 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:04.536631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080df7e-9118-4469-bc93-f68d468c3140 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:04.536753 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080df7e-9118-4469-bc93-f68d468c3140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:04.554421 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c080df7e-9118-4469-bc93-f68d468c3140 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:04.555641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c080df7e-9118-4469-bc93-f68d468c3140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:04.620849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1390/2779] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:20:04.632471 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:04.633543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:04.695397 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda6131-b667-4836-966a-3fcf528f21cd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:04.700293 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d1376-8d85-4482-883f-7b1290202572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:04.700953 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d1376-8d85-4482-883f-7b1290202572 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:04.701271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d1376-8d85-4482-883f-7b1290202572 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:04.701560 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d1376-8d85-4482-883f-7b1290202572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:04.711535 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d1376-8d85-4482-883f-7b1290202572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:04.713001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d1376-8d85-4482-883f-7b1290202572 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:04.723907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1390/2780] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:05.040229 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080df7e-9118-4469-bc93-f68d468c3140 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:05.043238 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c080df7e-9118-4469-bc93-f68d468c3140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:05.059378 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080df7e-9118-4469-bc93-f68d468c3140 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:05.064051 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:05.064915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:05.065306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:05.065488 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] PATH_INFO: `/v3/projects/e344d445db0c42a3a7f26efba5f734d8` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:05.066924 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e344d445db0c42a3a7f26efba5f734d8)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:05.068785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:05.127843 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e344d445db0c42a3a7f26efba5f734d8', 'request_id': 'req-1e3cf3af-e1a9-4554-a27b-256e4876661e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.136222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e344d445db0c42a3a7f26efba5f734d8', 'request_id': 'req-1e3cf3af-e1a9-4554-a27b-256e4876661e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.145559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e344d445db0c42a3a7f26efba5f734d8', 'request_id': 'req-1e3cf3af-e1a9-4554-a27b-256e4876661e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.161790 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] The token cache is being invalidate because project e344d445db0c42a3a7f26efba5f734d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:05.162195 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.162890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.164884 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1391/2781] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:05 2022] DELETE /identity/v3/projects/e344d445db0c42a3a7f26efba5f734d8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:05.172268 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d1376-8d85-4482-883f-7b1290202572 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:05.190648 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d1376-8d85-4482-883f-7b1290202572 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:05.287315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:05.288174 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:05.288463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:05.288742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] PATH_INFO: `/v3/users/7dbee1bce0e643acbaab424578fb8c32` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:05.290136 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dbee1bce0e643acbaab424578fb8c32)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:05.292639 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:05.351879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dbee1bce0e643acbaab424578fb8c32'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.357236 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:05.367982 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dbee1bce0e643acbaab424578fb8c32'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.377298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dbee1bce0e643acbaab424578fb8c32'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.385001 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dbee1bce0e643acbaab424578fb8c32'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.385721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-16bb64e6-7ff9-48bc-a6ab-5e36a34746ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dbee1bce0e643acbaab424578fb8c32'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.392141 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1391/2782] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:05 2022] DELETE /identity/v3/users/7dbee1bce0e643acbaab424578fb8c32 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:05.676347 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:05.678011 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:05.691081 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cf3af-e1a9-4554-a27b-256e4876661e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:05.736368 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:05.751592 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:05.752452 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:05.752737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:05.753019 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] PATH_INFO: `/v3/projects/b75a06eacf2f4b1bb2ca4c52dfe56cc1` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:05.754235 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75a06eacf2f4b1bb2ca4c52dfe56cc1)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:05.756130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:05.801744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75a06eacf2f4b1bb2ca4c52dfe56cc1', 'request_id': 'req-73451517-5963-468f-bf9c-72c7958c6a80'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.807744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75a06eacf2f4b1bb2ca4c52dfe56cc1', 'request_id': 'req-73451517-5963-468f-bf9c-72c7958c6a80'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.816778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75a06eacf2f4b1bb2ca4c52dfe56cc1', 'request_id': 'req-73451517-5963-468f-bf9c-72c7958c6a80'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.828836 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] The token cache is being invalidate because project b75a06eacf2f4b1bb2ca4c52dfe56cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:05.829146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.829827 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73451517-5963-468f-bf9c-72c7958c6a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:05.831408 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1392/2783] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:05 2022] DELETE /identity/v3/projects/b75a06eacf2f4b1bb2ca4c52dfe56cc1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:05.836625 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c0889-dd95-41bc-a8fd-e564e9c5cd95 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:05.836913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c0889-dd95-41bc-a8fd-e564e9c5cd95 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:05.837263 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c0889-dd95-41bc-a8fd-e564e9c5cd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:05.856615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-645c0889-dd95-41bc-a8fd-e564e9c5cd95 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:05.857882 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-645c0889-dd95-41bc-a8fd-e564e9c5cd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:05.922549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1392/2784] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:20:05.930614 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc173e-e731-4e44-91dc-e35d0485b837 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:05.930897 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc173e-e731-4e44-91dc-e35d0485b837 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:05.931272 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc173e-e731-4e44-91dc-e35d0485b837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:05.950704 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-20dc173e-e731-4e44-91dc-e35d0485b837 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:05.951939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-20dc173e-e731-4e44-91dc-e35d0485b837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:05.960418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1393/2785] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:05.965732 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c0889-dd95-41bc-a8fd-e564e9c5cd95 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:05.976552 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c0889-dd95-41bc-a8fd-e564e9c5cd95 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:05.981327 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699a922-d0c3-4773-9eb0-022d905b41e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:05.982053 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699a922-d0c3-4773-9eb0-022d905b41e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:05.982344 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699a922-d0c3-4773-9eb0-022d905b41e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:05.982645 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699a922-d0c3-4773-9eb0-022d905b41e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:05.983374 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699a922-d0c3-4773-9eb0-022d905b41e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:20:05.983684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699a922-d0c3-4773-9eb0-022d905b41e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:05.985343 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699a922-d0c3-4773-9eb0-022d905b41e2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:05.997820 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1393/2786] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:20:06.002535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc173e-e731-4e44-91dc-e35d0485b837 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:06.012880 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc173e-e731-4e44-91dc-e35d0485b837 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:06.017669 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9943d6b-9250-47f1-acc1-6f6c856dbdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:06.018319 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9943d6b-9250-47f1-acc1-6f6c856dbdde admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:06.018606 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9943d6b-9250-47f1-acc1-6f6c856dbdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:06.018885 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9943d6b-9250-47f1-acc1-6f6c856dbdde admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:06.019588 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9943d6b-9250-47f1-acc1-6f6c856dbdde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:06.021520 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9943d6b-9250-47f1-acc1-6f6c856dbdde admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:06.047747 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1394/2787] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:06 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:06.051949 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699a922-d0c3-4773-9eb0-022d905b41e2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:06.062723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699a922-d0c3-4773-9eb0-022d905b41e2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:06.120006 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9e296-f34a-46bd-a424-94c7e1fba50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:06.120670 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9e296-f34a-46bd-a424-94c7e1fba50e admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:06.120961 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9e296-f34a-46bd-a424-94c7e1fba50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:06.121280 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9e296-f34a-46bd-a424-94c7e1fba50e admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:06.121971 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9e296-f34a-46bd-a424-94c7e1fba50e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:06.123893 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9e296-f34a-46bd-a424-94c7e1fba50e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:06.144119 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1394/2788] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:06 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:06.148318 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9943d6b-9250-47f1-acc1-6f6c856dbdde admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:06.159170 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9943d6b-9250-47f1-acc1-6f6c856dbdde admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:06.163841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ede53-9f2a-4ba5-8021-f2faaf018dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:06.164487 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ede53-9f2a-4ba5-8021-f2faaf018dee admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:06.164772 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ede53-9f2a-4ba5-8021-f2faaf018dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:06.165049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ede53-9f2a-4ba5-8021-f2faaf018dee admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:06.165797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ede53-9f2a-4ba5-8021-f2faaf018dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:06.167926 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ede53-9f2a-4ba5-8021-f2faaf018dee admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:06.178216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1395/2789] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:06.182310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9e296-f34a-46bd-a424-94c7e1fba50e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:06.192723 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9e296-f34a-46bd-a424-94c7e1fba50e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:06.197672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292df951-0810-4ff3-ae03-b0fb1a9ae7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:06.198341 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292df951-0810-4ff3-ae03-b0fb1a9ae7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:06.198653 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292df951-0810-4ff3-ae03-b0fb1a9ae7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:06.198934 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292df951-0810-4ff3-ae03-b0fb1a9ae7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:06.199662 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292df951-0810-4ff3-ae03-b0fb1a9ae7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:06.201285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292df951-0810-4ff3-ae03-b0fb1a9ae7f0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:06.211303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1395/2790] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:20:06.215882 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ede53-9f2a-4ba5-8021-f2faaf018dee admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:06.226146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ede53-9f2a-4ba5-8021-f2faaf018dee admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:06.231742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b65f8d-40c0-4480-a5a3-6c8dc0d675f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:06.232638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65f8d-40c0-4480-a5a3-6c8dc0d675f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:06.232944 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65f8d-40c0-4480-a5a3-6c8dc0d675f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:06.233252 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65f8d-40c0-4480-a5a3-6c8dc0d675f6 admin admin] PATH_INFO: `/v3/projects/70a11575a36747b99cb6be64095dfa24/users/82193363d82e45b3b962b8ed902e5334/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:06.249102 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b65f8d-40c0-4480-a5a3-6c8dc0d675f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a11575a36747b99cb6be64095dfa24, user_id=82193363d82e45b3b962b8ed902e5334, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:06.251850 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b65f8d-40c0-4480-a5a3-6c8dc0d675f6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:06.265787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1396/2791] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:06 2022] PUT /identity/v3/projects/70a11575a36747b99cb6be64095dfa24/users/82193363d82e45b3b962b8ed902e5334/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:06.270960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85166b5-f1b3-4590-8436-f92f43230e68 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:06.271255 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85166b5-f1b3-4590-8436-f92f43230e68 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:06.271543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85166b5-f1b3-4590-8436-f92f43230e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:06.289666 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c85166b5-f1b3-4590-8436-f92f43230e68 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:06.290860 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c85166b5-f1b3-4590-8436-f92f43230e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:06.326060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:06.360828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1396/2792] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:06.370799 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b65f8d-40c0-4480-a5a3-6c8dc0d675f6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:06.372735 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d5b65f8d-40c0-4480-a5a3-6c8dc0d675f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:06.440559 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b65f8d-40c0-4480-a5a3-6c8dc0d675f6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:06.444475 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111749d-b0a4-4305-b40e-06f4fb020c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:06.445149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111749d-b0a4-4305-b40e-06f4fb020c75 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:06.445442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111749d-b0a4-4305-b40e-06f4fb020c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:06.445721 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111749d-b0a4-4305-b40e-06f4fb020c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:06.457546 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111749d-b0a4-4305-b40e-06f4fb020c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:06.459009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111749d-b0a4-4305-b40e-06f4fb020c75 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:06.470158 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1397/2793] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:07.106643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ee3b-4c50-4a8f-a17d-e3a502660809 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:07.107013 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ee3b-4c50-4a8f-a17d-e3a502660809 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:07.107405 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ee3b-4c50-4a8f-a17d-e3a502660809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:07.139636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-96e0ee3b-4c50-4a8f-a17d-e3a502660809 None None] MFA Rules not processed for user `82193363d82e45b3b962b8ed902e5334`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:07.141027 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-96e0ee3b-4c50-4a8f-a17d-e3a502660809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:07.216225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1397/2794] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:07.220732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111749d-b0a4-4305-b40e-06f4fb020c75 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:07.231045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111749d-b0a4-4305-b40e-06f4fb020c75 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:07.236188 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ce4ee-24ef-4a23-ba93-83e5ccd9dba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:07.236920 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ce4ee-24ef-4a23-ba93-83e5ccd9dba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:07.237306 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ce4ee-24ef-4a23-ba93-83e5ccd9dba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:07.237659 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ce4ee-24ef-4a23-ba93-83e5ccd9dba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:07.238458 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ce4ee-24ef-4a23-ba93-83e5ccd9dba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:07.240454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ce4ee-24ef-4a23-ba93-83e5ccd9dba3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:07.254381 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:07.272094 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1398/2795] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:07 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:07.277777 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ee3b-4c50-4a8f-a17d-e3a502660809 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:07.297879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ee3b-4c50-4a8f-a17d-e3a502660809 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:07.378258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff858f0-80de-4af5-8466-94404933cf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:07.378851 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff858f0-80de-4af5-8466-94404933cf76 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:07.379138 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff858f0-80de-4af5-8466-94404933cf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:07.379419 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff858f0-80de-4af5-8466-94404933cf76 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:07.380109 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff858f0-80de-4af5-8466-94404933cf76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:07.382080 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff858f0-80de-4af5-8466-94404933cf76 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:07.403965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1398/2796] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:07 2022] POST /identity/v3/users => generated 307 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:07.409329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ce4ee-24ef-4a23-ba93-83e5ccd9dba3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:07.428548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ce4ee-24ef-4a23-ba93-83e5ccd9dba3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:07.436266 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523ff1d-06fa-4666-8d42-d31a3fe67b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:07.437440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523ff1d-06fa-4666-8d42-d31a3fe67b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:07.437956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523ff1d-06fa-4666-8d42-d31a3fe67b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:07.438468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523ff1d-06fa-4666-8d42-d31a3fe67b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:07.439899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523ff1d-06fa-4666-8d42-d31a3fe67b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:07.442961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523ff1d-06fa-4666-8d42-d31a3fe67b0b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:07.464386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1399/2797] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:07.469222 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff858f0-80de-4af5-8466-94404933cf76 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:07.481276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff858f0-80de-4af5-8466-94404933cf76 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:07.486364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059866-32ad-4d92-8401-29be22650d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:07.487271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059866-32ad-4d92-8401-29be22650d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:07.487556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059866-32ad-4d92-8401-29be22650d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:07.487834 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059866-32ad-4d92-8401-29be22650d50 admin admin] PATH_INFO: `/v3/projects/08e7fa25cfe44a01b61c4e432db479ee/users/79cdd0e1e870405591b1476a04b1d4b7/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:07.506451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5059866-32ad-4d92-8401-29be22650d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e7fa25cfe44a01b61c4e432db479ee, user_id=79cdd0e1e870405591b1476a04b1d4b7, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:07.509437 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5059866-32ad-4d92-8401-29be22650d50 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:07.526700 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1399/2798] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:07 2022] PUT /identity/v3/projects/08e7fa25cfe44a01b61c4e432db479ee/users/79cdd0e1e870405591b1476a04b1d4b7/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:07.531594 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523ff1d-06fa-4666-8d42-d31a3fe67b0b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:07.548573 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523ff1d-06fa-4666-8d42-d31a3fe67b0b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:07.621032 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0775f14-128b-454d-bdf5-b4e462329d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:07.621746 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0775f14-128b-454d-bdf5-b4e462329d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:07.622036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0775f14-128b-454d-bdf5-b4e462329d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:07.622317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0775f14-128b-454d-bdf5-b4e462329d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:07.623079 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0775f14-128b-454d-bdf5-b4e462329d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:07.624711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0775f14-128b-454d-bdf5-b4e462329d2f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:07.636787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1400/2799] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:07.642954 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059866-32ad-4d92-8401-29be22650d50 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:07.654036 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059866-32ad-4d92-8401-29be22650d50 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:07.658822 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411c163-c089-4fa4-9361-9d143ac7daeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:07.659720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c163-c089-4fa4-9361-9d143ac7daeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:07.660005 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c163-c089-4fa4-9361-9d143ac7daeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:07.660288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c163-c089-4fa4-9361-9d143ac7daeb admin admin] PATH_INFO: `/v3/projects/08e7fa25cfe44a01b61c4e432db479ee/users/79cdd0e1e870405591b1476a04b1d4b7/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:07.662311 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411c163-c089-4fa4-9361-9d143ac7daeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e7fa25cfe44a01b61c4e432db479ee, user_id=79cdd0e1e870405591b1476a04b1d4b7, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:07.664967 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411c163-c089-4fa4-9361-9d143ac7daeb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:07.677897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1400/2800] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:07 2022] PUT /identity/v3/projects/08e7fa25cfe44a01b61c4e432db479ee/users/79cdd0e1e870405591b1476a04b1d4b7/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:08.399118 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4c477-e3f6-4b0c-bc53-fd34f71b4692 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:08.400004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4c477-e3f6-4b0c-bc53-fd34f71b4692 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:08.400004 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4c477-e3f6-4b0c-bc53-fd34f71b4692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:08.430049 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-46d4c477-e3f6-4b0c-bc53-fd34f71b4692 None None] MFA Rules not processed for user `79cdd0e1e870405591b1476a04b1d4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:08.431311 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-46d4c477-e3f6-4b0c-bc53-fd34f71b4692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:08.518609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1401/2801] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:08 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:20:09.966705 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411c163-c089-4fa4-9361-9d143ac7daeb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:10.000372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:10.054165 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411c163-c089-4fa4-9361-9d143ac7daeb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:10.059528 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:10.060421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:10.060707 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:10.060994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] PATH_INFO: `/v3/users/82193363d82e45b3b962b8ed902e5334` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:10.062295 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=82193363d82e45b3b962b8ed902e5334)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:10.064262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:10.129312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82193363d82e45b3b962b8ed902e5334'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.151395 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82193363d82e45b3b962b8ed902e5334'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.160345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82193363d82e45b3b962b8ed902e5334'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.169070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82193363d82e45b3b962b8ed902e5334'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.169978 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82193363d82e45b3b962b8ed902e5334'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.177026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1401/2802] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:09 2022] DELETE /identity/v3/users/82193363d82e45b3b962b8ed902e5334 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:20:10.383635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4c477-e3f6-4b0c-bc53-fd34f71b4692 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:10.396296 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4c477-e3f6-4b0c-bc53-fd34f71b4692 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:10.472597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:10.473563 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:10.473848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:10.474132 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] PATH_INFO: `/v3/projects/70a11575a36747b99cb6be64095dfa24` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:10.475390 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a11575a36747b99cb6be64095dfa24)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:10.478638 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:10.485525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:10.536575 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a11575a36747b99cb6be64095dfa24', 'request_id': 'req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.544447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a11575a36747b99cb6be64095dfa24', 'request_id': 'req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.554449 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a11575a36747b99cb6be64095dfa24', 'request_id': 'req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.567287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] The token cache is being invalidate because project 70a11575a36747b99cb6be64095dfa24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:10.567616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.568285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.570320 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1402/2803] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:10 2022] DELETE /identity/v3/projects/70a11575a36747b99cb6be64095dfa24 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:20:10.575468 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:10.576937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:10.591426 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3cbdc-43ad-4093-b722-7bb987b48daa admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:10.660616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:10.661570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:10.661911 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:10.662247 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] PATH_INFO: `/v3/users/79cdd0e1e870405591b1476a04b1d4b7` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:10.663647 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79cdd0e1e870405591b1476a04b1d4b7)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:10.666325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:10.720058 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:10.728615 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79cdd0e1e870405591b1476a04b1d4b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.747930 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79cdd0e1e870405591b1476a04b1d4b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.757885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79cdd0e1e870405591b1476a04b1d4b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.767996 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79cdd0e1e870405591b1476a04b1d4b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.768779 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-39030ef9-a8e0-475f-9f75-2e14f6c819b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79cdd0e1e870405591b1476a04b1d4b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:10.776253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1402/2804] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:10 2022] DELETE /identity/v3/users/79cdd0e1e870405591b1476a04b1d4b7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:20:10.985561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:10.987284 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:11.001749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c9b76-e19f-4a80-9cde-4b947c64af9e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:11.068904 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:11.069956 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.070342 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.070741 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] PATH_INFO: `/v3/projects/08e7fa25cfe44a01b61c4e432db479ee` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.072542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e7fa25cfe44a01b61c4e432db479ee)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:11.074763 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:11.130499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e7fa25cfe44a01b61c4e432db479ee', 'request_id': 'req-7f102117-8d2b-4768-ae7f-db102f1339dd'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:11.140064 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e7fa25cfe44a01b61c4e432db479ee', 'request_id': 'req-7f102117-8d2b-4768-ae7f-db102f1339dd'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:11.149795 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e7fa25cfe44a01b61c4e432db479ee', 'request_id': 'req-7f102117-8d2b-4768-ae7f-db102f1339dd'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:11.164165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] The token cache is being invalidate because project 08e7fa25cfe44a01b61c4e432db479ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:11.164702 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:11.165806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7f102117-8d2b-4768-ae7f-db102f1339dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:11.168948 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1403/2805] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:10 2022] DELETE /identity/v3/projects/08e7fa25cfe44a01b61c4e432db479ee => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:11.175793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71566e4-edb4-4cf2-b5d4-3caf001a5d5d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.176399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71566e4-edb4-4cf2-b5d4-3caf001a5d5d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.176399 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71566e4-edb4-4cf2-b5d4-3caf001a5d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.197320 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b71566e4-edb4-4cf2-b5d4-3caf001a5d5d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:11.198608 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b71566e4-edb4-4cf2-b5d4-3caf001a5d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:11.275657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1403/2806] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:11.288082 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a639a-f7da-48b8-81c5-16c63f3d15d2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.288363 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a639a-f7da-48b8-81c5-16c63f3d15d2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.288642 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a639a-f7da-48b8-81c5-16c63f3d15d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.311574 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-199a639a-f7da-48b8-81c5-16c63f3d15d2 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:11.312831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-199a639a-f7da-48b8-81c5-16c63f3d15d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:11.321883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1404/2807] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:11.326674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71566e4-edb4-4cf2-b5d4-3caf001a5d5d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:11.337079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71566e4-edb4-4cf2-b5d4-3caf001a5d5d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:11.342624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca72ed1-891c-4719-8534-570bea54d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:11.343306 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca72ed1-891c-4719-8534-570bea54d9ea admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.343594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca72ed1-891c-4719-8534-570bea54d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.343876 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca72ed1-891c-4719-8534-570bea54d9ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.344621 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca72ed1-891c-4719-8534-570bea54d9ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:20:11.344936 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca72ed1-891c-4719-8534-570bea54d9ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:11.346614 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca72ed1-891c-4719-8534-570bea54d9ea admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:11.360107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1404/2808] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:20:11.367111 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a639a-f7da-48b8-81c5-16c63f3d15d2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:11.383558 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a639a-f7da-48b8-81c5-16c63f3d15d2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:11.390562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af634de-0e4a-4bb4-8a93-871aee278bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:11.391731 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af634de-0e4a-4bb4-8a93-871aee278bce admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.392214 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af634de-0e4a-4bb4-8a93-871aee278bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.392600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af634de-0e4a-4bb4-8a93-871aee278bce admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.393445 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af634de-0e4a-4bb4-8a93-871aee278bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:11.395903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af634de-0e4a-4bb4-8a93-871aee278bce admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:11.422620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:11.439886 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1405/2809] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:11 2022] POST /identity/v3/projects => generated 393 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:11.444753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca72ed1-891c-4719-8534-570bea54d9ea admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:11.455727 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca72ed1-891c-4719-8534-570bea54d9ea admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:11.520254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37204ce7-cbc3-4a15-ac07-9bc9940c2f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:11.520957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37204ce7-cbc3-4a15-ac07-9bc9940c2f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.521288 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37204ce7-cbc3-4a15-ac07-9bc9940c2f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.521575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37204ce7-cbc3-4a15-ac07-9bc9940c2f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.522304 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37204ce7-cbc3-4a15-ac07-9bc9940c2f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:11.524287 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37204ce7-cbc3-4a15-ac07-9bc9940c2f2e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:11.551349 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1405/2810] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:11 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:11.556963 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af634de-0e4a-4bb4-8a93-871aee278bce admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:11.568834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af634de-0e4a-4bb4-8a93-871aee278bce admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:11.574230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50996ca-e760-4021-87ba-e7aca2d258d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:11.574915 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50996ca-e760-4021-87ba-e7aca2d258d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.575201 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50996ca-e760-4021-87ba-e7aca2d258d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.575570 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50996ca-e760-4021-87ba-e7aca2d258d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.576178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50996ca-e760-4021-87ba-e7aca2d258d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:11.577851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50996ca-e760-4021-87ba-e7aca2d258d4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:11.598519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1406/2811] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:11.602761 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37204ce7-cbc3-4a15-ac07-9bc9940c2f2e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:11.614780 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37204ce7-cbc3-4a15-ac07-9bc9940c2f2e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:11.619933 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d2e7a-3c04-45b0-894e-2da2fe010c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:11.620622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2e7a-3c04-45b0-894e-2da2fe010c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.620913 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2e7a-3c04-45b0-894e-2da2fe010c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.621238 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2e7a-3c04-45b0-894e-2da2fe010c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.621989 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d2e7a-3c04-45b0-894e-2da2fe010c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:11.623631 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d2e7a-3c04-45b0-894e-2da2fe010c6d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:11.635346 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1406/2812] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:11.639672 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50996ca-e760-4021-87ba-e7aca2d258d4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:11.650919 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50996ca-e760-4021-87ba-e7aca2d258d4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:11.655874 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202f84a-2abe-49c5-b34e-6df4e2c2688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:11.656789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202f84a-2abe-49c5-b34e-6df4e2c2688e admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.657077 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202f84a-2abe-49c5-b34e-6df4e2c2688e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.657387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202f84a-2abe-49c5-b34e-6df4e2c2688e admin admin] PATH_INFO: `/v3/projects/9a03f4bf01ee4c8ba3dd4bf1fb626bef/users/bc8d3167a6ae41adae305bef75db8eb2/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.675167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202f84a-2abe-49c5-b34e-6df4e2c2688e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a03f4bf01ee4c8ba3dd4bf1fb626bef, user_id=bc8d3167a6ae41adae305bef75db8eb2, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:11.677899 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202f84a-2abe-49c5-b34e-6df4e2c2688e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:11.693350 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1407/2813] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:11 2022] PUT /identity/v3/projects/9a03f4bf01ee4c8ba3dd4bf1fb626bef/users/bc8d3167a6ae41adae305bef75db8eb2/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:11.704868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711f85f-7adb-49ef-84c2-25da216afa44 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.705191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711f85f-7adb-49ef-84c2-25da216afa44 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.705478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711f85f-7adb-49ef-84c2-25da216afa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.735451 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d711f85f-7adb-49ef-84c2-25da216afa44 None None] MFA Rules not processed for user `bc8d3167a6ae41adae305bef75db8eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:11.736742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d711f85f-7adb-49ef-84c2-25da216afa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:11.753427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:11.841849 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1407/2814] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:11.847486 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202f84a-2abe-49c5-b34e-6df4e2c2688e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:11.935622 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202f84a-2abe-49c5-b34e-6df4e2c2688e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:11.943584 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1c76b-f2f8-44c0-a680-26b1c75e0a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:11.944515 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1c76b-f2f8-44c0-a680-26b1c75e0a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:11.945030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1c76b-f2f8-44c0-a680-26b1c75e0a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:11.945571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1c76b-f2f8-44c0-a680-26b1c75e0a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:11.947851 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1c76b-f2f8-44c0-a680-26b1c75e0a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:11.951711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1c76b-f2f8-44c0-a680-26b1c75e0a86 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:11.990473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1408/2815] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:11 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:11.994776 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711f85f-7adb-49ef-84c2-25da216afa44 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:12.006316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711f85f-7adb-49ef-84c2-25da216afa44 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:12.067209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61d9fc-2d38-4215-a3e6-2f98ebb3fcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:12.067927 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61d9fc-2d38-4215-a3e6-2f98ebb3fcee admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:12.068215 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61d9fc-2d38-4215-a3e6-2f98ebb3fcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:12.068496 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61d9fc-2d38-4215-a3e6-2f98ebb3fcee admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:12.069258 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61d9fc-2d38-4215-a3e6-2f98ebb3fcee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:12.071243 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61d9fc-2d38-4215-a3e6-2f98ebb3fcee admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:12.094144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1408/2816] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:11 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:20:12.098982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1c76b-f2f8-44c0-a680-26b1c75e0a86 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:12.111990 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1c76b-f2f8-44c0-a680-26b1c75e0a86 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:12.117407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df840e5f-6d33-4823-9efd-ebe66e997446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:12.118120 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df840e5f-6d33-4823-9efd-ebe66e997446 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:12.118509 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df840e5f-6d33-4823-9efd-ebe66e997446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:12.118831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df840e5f-6d33-4823-9efd-ebe66e997446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:12.120179 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df840e5f-6d33-4823-9efd-ebe66e997446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:12.121895 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df840e5f-6d33-4823-9efd-ebe66e997446 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:12.133191 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1409/2817] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:12.138104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61d9fc-2d38-4215-a3e6-2f98ebb3fcee admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:12.148998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61d9fc-2d38-4215-a3e6-2f98ebb3fcee admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:12.154113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48e6b0-c6db-4791-9c45-c7673feecd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:12.155026 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48e6b0-c6db-4791-9c45-c7673feecd0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:12.155315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48e6b0-c6db-4791-9c45-c7673feecd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:12.155602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48e6b0-c6db-4791-9c45-c7673feecd0c admin admin] PATH_INFO: `/v3/projects/35993f2288964641bd74bb65b2f5c5e1/users/3ea4ce36ab72400f8e7fe57a161f171e/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:12.171962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48e6b0-c6db-4791-9c45-c7673feecd0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=35993f2288964641bd74bb65b2f5c5e1, user_id=3ea4ce36ab72400f8e7fe57a161f171e, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:12.174694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48e6b0-c6db-4791-9c45-c7673feecd0c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:12.190031 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1409/2818] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:12 2022] PUT /identity/v3/projects/35993f2288964641bd74bb65b2f5c5e1/users/3ea4ce36ab72400f8e7fe57a161f171e/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:12.194310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df840e5f-6d33-4823-9efd-ebe66e997446 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:12.206196 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df840e5f-6d33-4823-9efd-ebe66e997446 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:12.275767 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9a2ca-7102-40f1-aecb-8c2c3bb82f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:12.276457 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9a2ca-7102-40f1-aecb-8c2c3bb82f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:12.276749 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9a2ca-7102-40f1-aecb-8c2c3bb82f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:12.277031 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9a2ca-7102-40f1-aecb-8c2c3bb82f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:12.277806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9a2ca-7102-40f1-aecb-8c2c3bb82f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:12.279435 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9a2ca-7102-40f1-aecb-8c2c3bb82f1f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:12.290193 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1410/2819] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:12.294581 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48e6b0-c6db-4791-9c45-c7673feecd0c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:12.305542 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48e6b0-c6db-4791-9c45-c7673feecd0c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:12.310965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b1b54-d956-4fc3-81b8-344dc9f04052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:12.311897 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b1b54-d956-4fc3-81b8-344dc9f04052 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:12.312183 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b1b54-d956-4fc3-81b8-344dc9f04052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:12.312463 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b1b54-d956-4fc3-81b8-344dc9f04052 admin admin] PATH_INFO: `/v3/projects/35993f2288964641bd74bb65b2f5c5e1/users/3ea4ce36ab72400f8e7fe57a161f171e/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:12.314504 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b1b54-d956-4fc3-81b8-344dc9f04052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35993f2288964641bd74bb65b2f5c5e1, user_id=3ea4ce36ab72400f8e7fe57a161f171e, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:12.317237 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b1b54-d956-4fc3-81b8-344dc9f04052 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:12.330353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1410/2820] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:12 2022] PUT /identity/v3/projects/35993f2288964641bd74bb65b2f5c5e1/users/3ea4ce36ab72400f8e7fe57a161f171e/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:12.342034 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0be51-32ed-42a3-ab0f-8db505f5b005 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:12.342327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0be51-32ed-42a3-ab0f-8db505f5b005 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:12.342611 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0be51-32ed-42a3-ab0f-8db505f5b005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:12.372130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d7b0be51-32ed-42a3-ab0f-8db505f5b005 None None] MFA Rules not processed for user `3ea4ce36ab72400f8e7fe57a161f171e`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:12.373423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d7b0be51-32ed-42a3-ab0f-8db505f5b005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:12.393386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:12.449197 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1411/2821] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:12.460094 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b1b54-d956-4fc3-81b8-344dc9f04052 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:12.461880 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-dd4b1b54-d956-4fc3-81b8-344dc9f04052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:12.527674 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b1b54-d956-4fc3-81b8-344dc9f04052 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:12.531466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76d370-0df3-4f0c-82bd-aec3b669f9ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:12.532132 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d370-0df3-4f0c-82bd-aec3b669f9ee service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:12.532416 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d370-0df3-4f0c-82bd-aec3b669f9ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:12.532697 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d370-0df3-4f0c-82bd-aec3b669f9ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:12.543335 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d370-0df3-4f0c-82bd-aec3b669f9ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:12.544821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d370-0df3-4f0c-82bd-aec3b669f9ee service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:12.555736 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1411/2822] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:12.611549 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0be51-32ed-42a3-ab0f-8db505f5b005 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:12.613325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d7b0be51-32ed-42a3-ab0f-8db505f5b005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:12.627641 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0be51-32ed-42a3-ab0f-8db505f5b005 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:12.631461 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f3f0e-3619-4bcc-9233-7fd87c163389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:12.632126 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3f0e-3619-4bcc-9233-7fd87c163389 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:12.632418 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3f0e-3619-4bcc-9233-7fd87c163389 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:12.632695 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3f0e-3619-4bcc-9233-7fd87c163389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:12.696348 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f3f0e-3619-4bcc-9233-7fd87c163389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:12.698635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f3f0e-3619-4bcc-9233-7fd87c163389 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:12.708829 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1412/2823] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:20:16.203824 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76d370-0df3-4f0c-82bd-aec3b669f9ee service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:16.214262 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:16.281990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76d370-0df3-4f0c-82bd-aec3b669f9ee service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:16.286742 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:16.287643 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:16.287929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:16.288208 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] PATH_INFO: `/v3/users/bc8d3167a6ae41adae305bef75db8eb2` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:16.289512 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8d3167a6ae41adae305bef75db8eb2)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:16.291515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:16.345549 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8d3167a6ae41adae305bef75db8eb2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:16.358605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8d3167a6ae41adae305bef75db8eb2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:16.367151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8d3167a6ae41adae305bef75db8eb2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:16.375090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8d3167a6ae41adae305bef75db8eb2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:16.375809 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8d3167a6ae41adae305bef75db8eb2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:16.382105 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1412/2824] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:16 2022] DELETE /identity/v3/users/bc8d3167a6ae41adae305bef75db8eb2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:16.387733 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3519934-37ce-4ce3-a8c2-262aadef43f3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:16.388591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3519934-37ce-4ce3-a8c2-262aadef43f3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:16.388789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3519934-37ce-4ce3-a8c2-262aadef43f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:16.407234 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c3519934-37ce-4ce3-a8c2-262aadef43f3 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:16.408492 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c3519934-37ce-4ce3-a8c2-262aadef43f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:16.504076 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1413/2825] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:16.517974 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:16.521646 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:16.606756 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df697265-46fd-44b2-b5e4-f45383a8724e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:16.611249 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9de78-c773-4f44-9a3e-8645e540179a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:16.612096 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9de78-c773-4f44-9a3e-8645e540179a service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:16.612545 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9de78-c773-4f44-9a3e-8645e540179a service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:16.612972 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9de78-c773-4f44-9a3e-8645e540179a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:16.626898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9de78-c773-4f44-9a3e-8645e540179a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:16.628386 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9de78-c773-4f44-9a3e-8645e540179a service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:16.640364 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1413/2826] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 19:20:16.952039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3519934-37ce-4ce3-a8c2-262aadef43f3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:16.953706 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c3519934-37ce-4ce3-a8c2-262aadef43f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:16.967387 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3519934-37ce-4ce3-a8c2-262aadef43f3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:16.972517 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:16.973440 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:16.973725 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:16.974035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] PATH_INFO: `/v3/projects/9a03f4bf01ee4c8ba3dd4bf1fb626bef` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:16.975427 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a03f4bf01ee4c8ba3dd4bf1fb626bef)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:16.977468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:17.047608 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a03f4bf01ee4c8ba3dd4bf1fb626bef', 'request_id': 'req-7e214225-a068-4e3e-8a86-3327c50c6622'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.049168 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:17.057539 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a03f4bf01ee4c8ba3dd4bf1fb626bef', 'request_id': 'req-7e214225-a068-4e3e-8a86-3327c50c6622'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.066436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a03f4bf01ee4c8ba3dd4bf1fb626bef', 'request_id': 'req-7e214225-a068-4e3e-8a86-3327c50c6622'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.078832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] The token cache is being invalidate because project 9a03f4bf01ee4c8ba3dd4bf1fb626bef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:17.079124 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.079784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.081375 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1414/2827] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:16 2022] DELETE /identity/v3/projects/9a03f4bf01ee4c8ba3dd4bf1fb626bef => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:20:17.085571 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9de78-c773-4f44-9a3e-8645e540179a service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:17.095665 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9de78-c773-4f44-9a3e-8645e540179a service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:17.125667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:17.162310 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:17.163191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:17.163485 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:17.163772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] PATH_INFO: `/v3/users/3ea4ce36ab72400f8e7fe57a161f171e` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:17.164994 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ea4ce36ab72400f8e7fe57a161f171e)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:17.167493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:17.219400 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ea4ce36ab72400f8e7fe57a161f171e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.232868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ea4ce36ab72400f8e7fe57a161f171e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.242921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ea4ce36ab72400f8e7fe57a161f171e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.253440 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ea4ce36ab72400f8e7fe57a161f171e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.254201 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4d14cb2-7d14-413e-b51d-3be07b6bdfbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ea4ce36ab72400f8e7fe57a161f171e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.260624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1414/2828] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:17 2022] DELETE /identity/v3/users/3ea4ce36ab72400f8e7fe57a161f171e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:17.606834 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:17.608439 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:17.625504 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e214225-a068-4e3e-8a86-3327c50c6622 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:17.692292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:17.693211 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:17.693499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:17.693778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] PATH_INFO: `/v3/projects/35993f2288964641bd74bb65b2f5c5e1` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:17.695056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35993f2288964641bd74bb65b2f5c5e1)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:17.697597 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:17.761246 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35993f2288964641bd74bb65b2f5c5e1', 'request_id': 'req-3d3a2c76-fef6-4af7-9e54-3d17276f1234'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.768310 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35993f2288964641bd74bb65b2f5c5e1', 'request_id': 'req-3d3a2c76-fef6-4af7-9e54-3d17276f1234'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.778770 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35993f2288964641bd74bb65b2f5c5e1', 'request_id': 'req-3d3a2c76-fef6-4af7-9e54-3d17276f1234'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.791099 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] The token cache is being invalidate because project 35993f2288964641bd74bb65b2f5c5e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:17.791374 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.792044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3d3a2c76-fef6-4af7-9e54-3d17276f1234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:17.793696 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1415/2829] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:17 2022] DELETE /identity/v3/projects/35993f2288964641bd74bb65b2f5c5e1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:17.798943 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9529-706a-440d-b37d-65368ad1cd8d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:17.799373 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9529-706a-440d-b37d-65368ad1cd8d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:17.799752 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9529-706a-440d-b37d-65368ad1cd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:17.819618 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-eadf9529-706a-440d-b37d-65368ad1cd8d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:17.820870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-eadf9529-706a-440d-b37d-65368ad1cd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:17.888205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1415/2830] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:20:17.896149 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d82ad-354c-4a01-84ab-04a41db2d9b1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:17.896535 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d82ad-354c-4a01-84ab-04a41db2d9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:17.896910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d82ad-354c-4a01-84ab-04a41db2d9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:17.916321 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-753d82ad-354c-4a01-84ab-04a41db2d9b1 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:17.917571 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-753d82ad-354c-4a01-84ab-04a41db2d9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:17.926464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1416/2831] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:17.930884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf9529-706a-440d-b37d-65368ad1cd8d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:17.942721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf9529-706a-440d-b37d-65368ad1cd8d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:17.948162 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20109d37-62ec-4186-be9e-2e4fb84c0294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:17.948844 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20109d37-62ec-4186-be9e-2e4fb84c0294 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:17.949159 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20109d37-62ec-4186-be9e-2e4fb84c0294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:17.949445 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20109d37-62ec-4186-be9e-2e4fb84c0294 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:17.950205 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20109d37-62ec-4186-be9e-2e4fb84c0294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:20:17.950525 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20109d37-62ec-4186-be9e-2e4fb84c0294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:17.952182 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20109d37-62ec-4186-be9e-2e4fb84c0294 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:17.965853 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1416/2832] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:20:17.970613 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d82ad-354c-4a01-84ab-04a41db2d9b1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:17.981024 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d82ad-354c-4a01-84ab-04a41db2d9b1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:17.986001 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f178c-1806-4174-8ca2-761f9ed77942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:17.986680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f178c-1806-4174-8ca2-761f9ed77942 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:17.986971 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f178c-1806-4174-8ca2-761f9ed77942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:17.987255 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f178c-1806-4174-8ca2-761f9ed77942 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:17.987998 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f178c-1806-4174-8ca2-761f9ed77942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:17.990018 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f178c-1806-4174-8ca2-761f9ed77942 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.018385 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1417/2833] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:17 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:20:18.022758 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20109d37-62ec-4186-be9e-2e4fb84c0294 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.033871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20109d37-62ec-4186-be9e-2e4fb84c0294 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.068116 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:18.097474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c7ad4-51d5-4966-8738-ff0bb51d9c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.098149 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c7ad4-51d5-4966-8738-ff0bb51d9c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.098433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c7ad4-51d5-4966-8738-ff0bb51d9c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.098735 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c7ad4-51d5-4966-8738-ff0bb51d9c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.099442 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c7ad4-51d5-4966-8738-ff0bb51d9c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.101410 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c7ad4-51d5-4966-8738-ff0bb51d9c19 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.123113 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1417/2834] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:18 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:20:18.127890 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f178c-1806-4174-8ca2-761f9ed77942 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.139317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f178c-1806-4174-8ca2-761f9ed77942 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.147275 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d21535-abc6-4ae2-960f-3c05ce0dcd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.147982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d21535-abc6-4ae2-960f-3c05ce0dcd40 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.148257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d21535-abc6-4ae2-960f-3c05ce0dcd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.148536 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d21535-abc6-4ae2-960f-3c05ce0dcd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.149305 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d21535-abc6-4ae2-960f-3c05ce0dcd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.151104 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d21535-abc6-4ae2-960f-3c05ce0dcd40 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.167591 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1418/2835] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:18.172924 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c7ad4-51d5-4966-8738-ff0bb51d9c19 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.184298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c7ad4-51d5-4966-8738-ff0bb51d9c19 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.189224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb79452-c09a-498b-b691-c31d84de1074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.189902 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb79452-c09a-498b-b691-c31d84de1074 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.190191 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb79452-c09a-498b-b691-c31d84de1074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.190474 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb79452-c09a-498b-b691-c31d84de1074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.191225 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb79452-c09a-498b-b691-c31d84de1074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.192907 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb79452-c09a-498b-b691-c31d84de1074 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.205594 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1418/2836] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:18.209560 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d21535-abc6-4ae2-960f-3c05ce0dcd40 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.220734 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d21535-abc6-4ae2-960f-3c05ce0dcd40 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.225612 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2dce3-0794-4932-b1e2-bc4f2b6a5f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.226529 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2dce3-0794-4932-b1e2-bc4f2b6a5f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.226811 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2dce3-0794-4932-b1e2-bc4f2b6a5f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.227086 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2dce3-0794-4932-b1e2-bc4f2b6a5f21 admin admin] PATH_INFO: `/v3/projects/45cf867915eb41ddae3c98e44ee4d5bf/users/7462ca95115c4fdf8705e709bd91bdc2/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.243122 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2dce3-0794-4932-b1e2-bc4f2b6a5f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45cf867915eb41ddae3c98e44ee4d5bf, user_id=7462ca95115c4fdf8705e709bd91bdc2, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.246359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2dce3-0794-4932-b1e2-bc4f2b6a5f21 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.264044 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1419/2837] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:18 2022] PUT /identity/v3/projects/45cf867915eb41ddae3c98e44ee4d5bf/users/7462ca95115c4fdf8705e709bd91bdc2/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:18.277493 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4051c7c-e7d7-4337-a75f-27c539531165 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.278303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4051c7c-e7d7-4337-a75f-27c539531165 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.278303 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4051c7c-e7d7-4337-a75f-27c539531165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.307160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a4051c7c-e7d7-4337-a75f-27c539531165 None None] MFA Rules not processed for user `7462ca95115c4fdf8705e709bd91bdc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:18.308422 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a4051c7c-e7d7-4337-a75f-27c539531165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:18.382433 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1419/2838] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:18.438621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2dce3-0794-4932-b1e2-bc4f2b6a5f21 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.444492 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:18.507471 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2dce3-0794-4932-b1e2-bc4f2b6a5f21 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.512257 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c24ea-d6e8-496b-a7b0-5c00d38ef7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.512921 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c24ea-d6e8-496b-a7b0-5c00d38ef7e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.513262 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c24ea-d6e8-496b-a7b0-5c00d38ef7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.513530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c24ea-d6e8-496b-a7b0-5c00d38ef7e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.514236 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c24ea-d6e8-496b-a7b0-5c00d38ef7e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.516165 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c24ea-d6e8-496b-a7b0-5c00d38ef7e0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.544253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1420/2839] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:18 2022] POST /identity/v3/projects => generated 401 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:18.548692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4051c7c-e7d7-4337-a75f-27c539531165 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.561359 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4051c7c-e7d7-4337-a75f-27c539531165 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.642633 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92efd01-8e16-4dfe-a72c-f415ea6320d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.643312 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92efd01-8e16-4dfe-a72c-f415ea6320d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.643595 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92efd01-8e16-4dfe-a72c-f415ea6320d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.643869 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92efd01-8e16-4dfe-a72c-f415ea6320d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.644575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92efd01-8e16-4dfe-a72c-f415ea6320d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.646543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92efd01-8e16-4dfe-a72c-f415ea6320d0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.669270 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1420/2840] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:18 2022] POST /identity/v3/users => generated 307 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:18.674256 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c24ea-d6e8-496b-a7b0-5c00d38ef7e0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.685711 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c24ea-d6e8-496b-a7b0-5c00d38ef7e0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.691377 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbce33-3cbf-4129-803f-1f2198a46117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.691986 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbce33-3cbf-4129-803f-1f2198a46117 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.692269 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbce33-3cbf-4129-803f-1f2198a46117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.692548 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbce33-3cbf-4129-803f-1f2198a46117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.693309 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbce33-3cbf-4129-803f-1f2198a46117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.694962 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbce33-3cbf-4129-803f-1f2198a46117 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.707800 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1421/2841] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:18.712388 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92efd01-8e16-4dfe-a72c-f415ea6320d0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.723667 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92efd01-8e16-4dfe-a72c-f415ea6320d0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.728570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d740c4-fce1-4b3e-9bd4-6f11cd48a5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.729540 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d740c4-fce1-4b3e-9bd4-6f11cd48a5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.729821 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d740c4-fce1-4b3e-9bd4-6f11cd48a5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.730097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d740c4-fce1-4b3e-9bd4-6f11cd48a5fc admin admin] PATH_INFO: `/v3/projects/28e10e31c14f4ae7a0356f1295651949/users/5b1a4512e6724d6489efc444f0aa7729/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.747908 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d740c4-fce1-4b3e-9bd4-6f11cd48a5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e10e31c14f4ae7a0356f1295651949, user_id=5b1a4512e6724d6489efc444f0aa7729, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.750602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d740c4-fce1-4b3e-9bd4-6f11cd48a5fc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.764778 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1421/2842] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:18 2022] PUT /identity/v3/projects/28e10e31c14f4ae7a0356f1295651949/users/5b1a4512e6724d6489efc444f0aa7729/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:18.768891 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbce33-3cbf-4129-803f-1f2198a46117 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.779871 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbce33-3cbf-4129-803f-1f2198a46117 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.836190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195b7cb-bc41-4243-8dd6-525ebad53c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.836856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b7cb-bc41-4243-8dd6-525ebad53c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.837178 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b7cb-bc41-4243-8dd6-525ebad53c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.837463 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b7cb-bc41-4243-8dd6-525ebad53c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.838798 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195b7cb-bc41-4243-8dd6-525ebad53c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.840391 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195b7cb-bc41-4243-8dd6-525ebad53c50 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.850748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1422/2843] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 19:20:18.855546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d740c4-fce1-4b3e-9bd4-6f11cd48a5fc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:18.866421 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d740c4-fce1-4b3e-9bd4-6f11cd48a5fc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:18.872065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1005f23-a4fb-49b4-89c5-d9af487faadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:18.872957 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1005f23-a4fb-49b4-89c5-d9af487faadb admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.873273 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1005f23-a4fb-49b4-89c5-d9af487faadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.873554 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1005f23-a4fb-49b4-89c5-d9af487faadb admin admin] PATH_INFO: `/v3/projects/28e10e31c14f4ae7a0356f1295651949/users/5b1a4512e6724d6489efc444f0aa7729/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.875543 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1005f23-a4fb-49b4-89c5-d9af487faadb admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e10e31c14f4ae7a0356f1295651949, user_id=5b1a4512e6724d6489efc444f0aa7729, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:18.878241 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1005f23-a4fb-49b4-89c5-d9af487faadb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:18.881649 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:18.891260 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1422/2844] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:18 2022] PUT /identity/v3/projects/28e10e31c14f4ae7a0356f1295651949/users/5b1a4512e6724d6489efc444f0aa7729/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:18.904875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2826-b93a-4e0b-a90e-4d38d05c8b76 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:18.905203 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2826-b93a-4e0b-a90e-4d38d05c8b76 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:18.905489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2826-b93a-4e0b-a90e-4d38d05c8b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:18.935068 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8f1f2826-b93a-4e0b-a90e-4d38d05c8b76 None None] MFA Rules not processed for user `5b1a4512e6724d6489efc444f0aa7729`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:18.936454 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8f1f2826-b93a-4e0b-a90e-4d38d05c8b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:19.005134 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:19.010093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1423/2845] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:19.021218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1005f23-a4fb-49b4-89c5-d9af487faadb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:19.023123 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f1005f23-a4fb-49b4-89c5-d9af487faadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:19.082120 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1005f23-a4fb-49b4-89c5-d9af487faadb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:19.085859 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e27c5-a98e-401f-ac42-141ab97755ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:19.086501 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e27c5-a98e-401f-ac42-141ab97755ed service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:19.086791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e27c5-a98e-401f-ac42-141ab97755ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:19.087070 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e27c5-a98e-401f-ac42-141ab97755ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:19.096797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e27c5-a98e-401f-ac42-141ab97755ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:19.098275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e27c5-a98e-401f-ac42-141ab97755ed service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:19.108413 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1423/2846] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:19.184442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f2826-b93a-4e0b-a90e-4d38d05c8b76 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:19.194483 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f2826-b93a-4e0b-a90e-4d38d05c8b76 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:19.198856 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094118ba-2350-434f-b213-b420a45ed35d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:19.199522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094118ba-2350-434f-b213-b420a45ed35d service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:19.199810 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094118ba-2350-434f-b213-b420a45ed35d service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:19.200093 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094118ba-2350-434f-b213-b420a45ed35d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:19.261684 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094118ba-2350-434f-b213-b420a45ed35d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:19.263404 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094118ba-2350-434f-b213-b420a45ed35d service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:19.273607 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1424/2847] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:23.363102 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e27c5-a98e-401f-ac42-141ab97755ed service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:23.431730 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e27c5-a98e-401f-ac42-141ab97755ed service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:23.436301 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:23.437209 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:23.437642 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:23.437929 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] PATH_INFO: `/v3/users/7462ca95115c4fdf8705e709bd91bdc2` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:23.439176 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7462ca95115c4fdf8705e709bd91bdc2)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:23.441152 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:23.491090 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7462ca95115c4fdf8705e709bd91bdc2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:23.503044 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7462ca95115c4fdf8705e709bd91bdc2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:23.510793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7462ca95115c4fdf8705e709bd91bdc2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:23.518773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7462ca95115c4fdf8705e709bd91bdc2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:23.519601 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7462ca95115c4fdf8705e709bd91bdc2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:23.526298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1424/2848] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:23 2022] DELETE /identity/v3/users/7462ca95115c4fdf8705e709bd91bdc2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:23.530878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672455e-650f-47ea-857c-af66f17953bc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:23.531760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672455e-650f-47ea-857c-af66f17953bc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:23.531760 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672455e-650f-47ea-857c-af66f17953bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:23.549395 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c672455e-650f-47ea-857c-af66f17953bc None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:23.550644 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c672455e-650f-47ea-857c-af66f17953bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:23.613344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1425/2849] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:23.623057 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:23.624611 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:23.647557 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:23.692520 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f274b-31aa-42bc-b4b0-48d971a49db5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:23.696720 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df7966-b0cd-4ae7-9c35-cb0af76147e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:23.698180 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df7966-b0cd-4ae7-9c35-cb0af76147e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:23.698462 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df7966-b0cd-4ae7-9c35-cb0af76147e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:23.698738 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df7966-b0cd-4ae7-9c35-cb0af76147e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:23.709253 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df7966-b0cd-4ae7-9c35-cb0af76147e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:23.710721 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df7966-b0cd-4ae7-9c35-cb0af76147e1 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:23.721677 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1425/2850] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:24.014354 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672455e-650f-47ea-857c-af66f17953bc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:24.015961 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c672455e-650f-47ea-857c-af66f17953bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:24.030976 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672455e-650f-47ea-857c-af66f17953bc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:24.035784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:24.036680 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:24.037146 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:24.037436 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] PATH_INFO: `/v3/projects/45cf867915eb41ddae3c98e44ee4d5bf` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:24.038489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45cf867915eb41ddae3c98e44ee4d5bf)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:24.040425 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:24.093357 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45cf867915eb41ddae3c98e44ee4d5bf', 'request_id': 'req-ae5a7ee6-13d0-44a7-be59-cdc403a26980'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.099828 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45cf867915eb41ddae3c98e44ee4d5bf', 'request_id': 'req-ae5a7ee6-13d0-44a7-be59-cdc403a26980'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.110964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45cf867915eb41ddae3c98e44ee4d5bf', 'request_id': 'req-ae5a7ee6-13d0-44a7-be59-cdc403a26980'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.124883 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] The token cache is being invalidate because project 45cf867915eb41ddae3c98e44ee4d5bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:24.125190 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.125864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.127442 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1426/2851] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:24 2022] DELETE /identity/v3/projects/45cf867915eb41ddae3c98e44ee4d5bf => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:24.131793 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df7966-b0cd-4ae7-9c35-cb0af76147e1 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:24.142644 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df7966-b0cd-4ae7-9c35-cb0af76147e1 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:24.201075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:24.201981 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:24.202271 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:24.202551 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] PATH_INFO: `/v3/users/5b1a4512e6724d6489efc444f0aa7729` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:24.203785 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1a4512e6724d6489efc444f0aa7729)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:24.205772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:24.256515 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1a4512e6724d6489efc444f0aa7729'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.270124 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1a4512e6724d6489efc444f0aa7729'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.280829 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1a4512e6724d6489efc444f0aa7729'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.285673 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1a4512e6724d6489efc444f0aa7729'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.286353 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-15cdab19-d7aa-4072-ac72-1d201f891464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1a4512e6724d6489efc444f0aa7729'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.292636 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1426/2852] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:24 2022] DELETE /identity/v3/users/5b1a4512e6724d6489efc444f0aa7729 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:24.594036 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:24.595526 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:24.611748 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a7ee6-13d0-44a7-be59-cdc403a26980 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:24.640685 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:24.685369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:24.686247 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:24.686542 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:24.686814 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] PATH_INFO: `/v3/projects/28e10e31c14f4ae7a0356f1295651949` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:24.688039 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e10e31c14f4ae7a0356f1295651949)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:24.689988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:24.744552 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e10e31c14f4ae7a0356f1295651949', 'request_id': 'req-467d9eb8-118d-45bd-b1f5-b630175cc681'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.750940 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e10e31c14f4ae7a0356f1295651949', 'request_id': 'req-467d9eb8-118d-45bd-b1f5-b630175cc681'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.758742 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e10e31c14f4ae7a0356f1295651949', 'request_id': 'req-467d9eb8-118d-45bd-b1f5-b630175cc681'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.769396 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] The token cache is being invalidate because project 28e10e31c14f4ae7a0356f1295651949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:24.769616 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.770286 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-467d9eb8-118d-45bd-b1f5-b630175cc681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:24.771830 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1427/2853] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:24 2022] DELETE /identity/v3/projects/28e10e31c14f4ae7a0356f1295651949 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:24.776808 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8b86-6fcf-4001-9415-f0f4ea720d87 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:24.777334 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8b86-6fcf-4001-9415-f0f4ea720d87 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:24.777439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8b86-6fcf-4001-9415-f0f4ea720d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:24.796852 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-00ec8b86-6fcf-4001-9415-f0f4ea720d87 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:24.798118 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-00ec8b86-6fcf-4001-9415-f0f4ea720d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:24.833354 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:24.862147 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1427/2854] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:24.870637 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52dc3a-510f-4a54-bceb-062983c9b53f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:24.870979 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52dc3a-510f-4a54-bceb-062983c9b53f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:24.871324 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52dc3a-510f-4a54-bceb-062983c9b53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:24.891258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7a52dc3a-510f-4a54-bceb-062983c9b53f None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:24.892950 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7a52dc3a-510f-4a54-bceb-062983c9b53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:24.901175 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1428/2855] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:24.905559 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec8b86-6fcf-4001-9415-f0f4ea720d87 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:24.916506 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec8b86-6fcf-4001-9415-f0f4ea720d87 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:24.921431 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f5ad-d6c3-43a0-ba13-b3af2c046b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:24.922323 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f5ad-d6c3-43a0-ba13-b3af2c046b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:24.922701 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f5ad-d6c3-43a0-ba13-b3af2c046b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:24.922984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f5ad-d6c3-43a0-ba13-b3af2c046b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:24.923733 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50f5ad-d6c3-43a0-ba13-b3af2c046b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:20:24.924043 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50f5ad-d6c3-43a0-ba13-b3af2c046b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:24.925722 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50f5ad-d6c3-43a0-ba13-b3af2c046b3a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:24.938960 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1428/2856] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:20:24.943620 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52dc3a-510f-4a54-bceb-062983c9b53f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:24.954002 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52dc3a-510f-4a54-bceb-062983c9b53f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:24.958878 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6866a-5a5b-4541-94c9-1f702a76a4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:24.959555 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6866a-5a5b-4541-94c9-1f702a76a4db admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:24.959846 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6866a-5a5b-4541-94c9-1f702a76a4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:24.960129 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6866a-5a5b-4541-94c9-1f702a76a4db admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:24.960852 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6866a-5a5b-4541-94c9-1f702a76a4db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:24.963352 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6866a-5a5b-4541-94c9-1f702a76a4db admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:24.991775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1429/2857] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:24 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:24.996037 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f5ad-d6c3-43a0-ba13-b3af2c046b3a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.006724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f5ad-d6c3-43a0-ba13-b3af2c046b3a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.068024 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71937d-121f-4637-8a39-728d53149a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.068624 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71937d-121f-4637-8a39-728d53149a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.068912 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71937d-121f-4637-8a39-728d53149a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.069233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71937d-121f-4637-8a39-728d53149a00 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.069939 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d71937d-121f-4637-8a39-728d53149a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.071888 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d71937d-121f-4637-8a39-728d53149a00 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.093059 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1429/2858] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:24 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:25.097252 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6866a-5a5b-4541-94c9-1f702a76a4db admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.108859 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6866a-5a5b-4541-94c9-1f702a76a4db admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.113751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19080-49ff-4987-9157-4acc29bbad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.114420 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19080-49ff-4987-9157-4acc29bbad80 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.114708 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19080-49ff-4987-9157-4acc29bbad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.114988 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19080-49ff-4987-9157-4acc29bbad80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.115718 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee19080-49ff-4987-9157-4acc29bbad80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.117369 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee19080-49ff-4987-9157-4acc29bbad80 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.130621 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1430/2859] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:25.134796 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71937d-121f-4637-8a39-728d53149a00 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.145546 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71937d-121f-4637-8a39-728d53149a00 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.151111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d715f-78f1-417c-9581-7da4816d04df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.152107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d715f-78f1-417c-9581-7da4816d04df admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.152394 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d715f-78f1-417c-9581-7da4816d04df admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.152672 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d715f-78f1-417c-9581-7da4816d04df admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.153457 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d715f-78f1-417c-9581-7da4816d04df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.155097 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d715f-78f1-417c-9581-7da4816d04df admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.167015 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1430/2860] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:25.171498 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19080-49ff-4987-9157-4acc29bbad80 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.182317 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19080-49ff-4987-9157-4acc29bbad80 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.187227 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf69690-a206-497c-83a2-cb00db9a27f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.188131 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf69690-a206-497c-83a2-cb00db9a27f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.188462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf69690-a206-497c-83a2-cb00db9a27f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.188743 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf69690-a206-497c-83a2-cb00db9a27f4 admin admin] PATH_INFO: `/v3/projects/9da4240ffd66455cbdba47b9ac3b96bc/users/d8b664591a524ea78f954494f1d75c48/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.207519 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf69690-a206-497c-83a2-cb00db9a27f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9da4240ffd66455cbdba47b9ac3b96bc, user_id=d8b664591a524ea78f954494f1d75c48, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.210797 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf69690-a206-497c-83a2-cb00db9a27f4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.227939 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1431/2861] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:25 2022] PUT /identity/v3/projects/9da4240ffd66455cbdba47b9ac3b96bc/users/d8b664591a524ea78f954494f1d75c48/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:20:25.239873 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742a77a-a2f6-4fc0-82c8-d4b77f852dab None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.240219 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742a77a-a2f6-4fc0-82c8-d4b77f852dab None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.240534 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742a77a-a2f6-4fc0-82c8-d4b77f852dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.274290 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6742a77a-a2f6-4fc0-82c8-d4b77f852dab None None] MFA Rules not processed for user `d8b664591a524ea78f954494f1d75c48`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:25.276111 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6742a77a-a2f6-4fc0-82c8-d4b77f852dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:25.352480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1431/2862] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:25.357561 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf69690-a206-497c-83a2-cb00db9a27f4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.413710 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:25.430583 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf69690-a206-497c-83a2-cb00db9a27f4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.438263 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f061cf4-3ead-49f9-a34d-ec6ff001e6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.439398 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f061cf4-3ead-49f9-a34d-ec6ff001e6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.439947 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f061cf4-3ead-49f9-a34d-ec6ff001e6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.440451 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f061cf4-3ead-49f9-a34d-ec6ff001e6a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.441841 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f061cf4-3ead-49f9-a34d-ec6ff001e6a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.445499 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f061cf4-3ead-49f9-a34d-ec6ff001e6a7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.493364 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1432/2863] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:25 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:20:25.497855 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742a77a-a2f6-4fc0-82c8-d4b77f852dab None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.509925 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742a77a-a2f6-4fc0-82c8-d4b77f852dab None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.547541 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:25.576425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086552e-6657-45de-a98a-46399013543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.577143 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086552e-6657-45de-a98a-46399013543d admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.577439 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086552e-6657-45de-a98a-46399013543d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.577718 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086552e-6657-45de-a98a-46399013543d admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.578436 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086552e-6657-45de-a98a-46399013543d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.580791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086552e-6657-45de-a98a-46399013543d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.603737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1432/2864] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:25 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:25.608226 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f061cf4-3ead-49f9-a34d-ec6ff001e6a7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.620628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f061cf4-3ead-49f9-a34d-ec6ff001e6a7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.625629 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfb188-7d1e-4725-adb6-38be7e95e85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.626314 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfb188-7d1e-4725-adb6-38be7e95e85c admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.626635 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfb188-7d1e-4725-adb6-38be7e95e85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.626918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfb188-7d1e-4725-adb6-38be7e95e85c admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.627662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfb188-7d1e-4725-adb6-38be7e95e85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.629340 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfb188-7d1e-4725-adb6-38be7e95e85c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.643125 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1433/2865] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:25.647692 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086552e-6657-45de-a98a-46399013543d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.659767 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086552e-6657-45de-a98a-46399013543d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.664706 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb06046-584a-42fd-8d26-9294c1b2145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.665680 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb06046-584a-42fd-8d26-9294c1b2145c admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.665968 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb06046-584a-42fd-8d26-9294c1b2145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.666254 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb06046-584a-42fd-8d26-9294c1b2145c admin admin] PATH_INFO: `/v3/projects/615599002a9b4c3ab760c3587014cfd7/users/a952a2d23fa742aea7173286ef2fd0fd/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.683475 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb06046-584a-42fd-8d26-9294c1b2145c admin admin] RBAC: Authorizing `identity:create_grant(project_id=615599002a9b4c3ab760c3587014cfd7, user_id=a952a2d23fa742aea7173286ef2fd0fd, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.686211 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb06046-584a-42fd-8d26-9294c1b2145c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.701068 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1433/2866] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:25 2022] PUT /identity/v3/projects/615599002a9b4c3ab760c3587014cfd7/users/a952a2d23fa742aea7173286ef2fd0fd/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:25.705460 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfb188-7d1e-4725-adb6-38be7e95e85c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.716209 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfb188-7d1e-4725-adb6-38be7e95e85c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.778832 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0593f-6c75-42c0-9607-2866501a3818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.779873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0593f-6c75-42c0-9607-2866501a3818 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.779873 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0593f-6c75-42c0-9607-2866501a3818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.780166 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0593f-6c75-42c0-9607-2866501a3818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.780831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0593f-6c75-42c0-9607-2866501a3818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.782587 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0593f-6c75-42c0-9607-2866501a3818 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.793015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1434/2867] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:25.797298 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb06046-584a-42fd-8d26-9294c1b2145c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.808773 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb06046-584a-42fd-8d26-9294c1b2145c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:25.814107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a3a4-de25-4285-a3ec-42ac6a2626c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:25.814990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a3a4-de25-4285-a3ec-42ac6a2626c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.815276 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a3a4-de25-4285-a3ec-42ac6a2626c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.815556 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a3a4-de25-4285-a3ec-42ac6a2626c9 admin admin] PATH_INFO: `/v3/projects/615599002a9b4c3ab760c3587014cfd7/users/a952a2d23fa742aea7173286ef2fd0fd/roles/e089a0d6040040c4bcf143a680488e3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.817582 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18a3a4-de25-4285-a3ec-42ac6a2626c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615599002a9b4c3ab760c3587014cfd7, user_id=a952a2d23fa742aea7173286ef2fd0fd, role_id=e089a0d6040040c4bcf143a680488e3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:25.820216 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18a3a4-de25-4285-a3ec-42ac6a2626c9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:25.835660 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1434/2868] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:25 2022] PUT /identity/v3/projects/615599002a9b4c3ab760c3587014cfd7/users/a952a2d23fa742aea7173286ef2fd0fd/roles/e089a0d6040040c4bcf143a680488e3d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:25.846030 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd0c43-8cda-44d1-ba36-8d1468a3b658 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:25.846327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd0c43-8cda-44d1-ba36-8d1468a3b658 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:25.846609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd0c43-8cda-44d1-ba36-8d1468a3b658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:25.876660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-16cd0c43-8cda-44d1-ba36-8d1468a3b658 None None] MFA Rules not processed for user `a952a2d23fa742aea7173286ef2fd0fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:25.878056 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-16cd0c43-8cda-44d1-ba36-8d1468a3b658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:25.950779 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1435/2869] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:25.962121 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a3a4-de25-4285-a3ec-42ac6a2626c9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:25.963885 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0e18a3a4-de25-4285-a3ec-42ac6a2626c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:26.023532 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a3a4-de25-4285-a3ec-42ac6a2626c9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:26.027409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271b4bb-a99b-4354-86c2-2856c7c7e457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:26.028081 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271b4bb-a99b-4354-86c2-2856c7c7e457 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:26.028369 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271b4bb-a99b-4354-86c2-2856c7c7e457 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:26.028650 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271b4bb-a99b-4354-86c2-2856c7c7e457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:26.091947 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271b4bb-a99b-4354-86c2-2856c7c7e457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:26.093361 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271b4bb-a99b-4354-86c2-2856c7c7e457 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:26.103775 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1435/2870] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:28.648015 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd0c43-8cda-44d1-ba36-8d1468a3b658 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:28.694610 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:28.716600 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd0c43-8cda-44d1-ba36-8d1468a3b658 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:28.721285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:28.722167 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:28.722462 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:28.722744 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] PATH_INFO: `/v3/users/d8b664591a524ea78f954494f1d75c48` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:28.723966 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b664591a524ea78f954494f1d75c48)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:28.725947 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:28.777010 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b664591a524ea78f954494f1d75c48'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:28.789224 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b664591a524ea78f954494f1d75c48'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:28.797849 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b664591a524ea78f954494f1d75c48'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:28.798667 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b664591a524ea78f954494f1d75c48'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:28.807737 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b664591a524ea78f954494f1d75c48'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:28.813965 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1436/2871] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:28 2022] DELETE /identity/v3/users/d8b664591a524ea78f954494f1d75c48 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:28.818513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc364-a214-4b9d-991c-f1ce6afdf30d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:28.819022 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc364-a214-4b9d-991c-f1ce6afdf30d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:28.819127 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc364-a214-4b9d-991c-f1ce6afdf30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:28.837652 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ffdfc364-a214-4b9d-991c-f1ce6afdf30d None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:28.838898 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ffdfc364-a214-4b9d-991c-f1ce6afdf30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:28.880079 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:28.902938 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1436/2872] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:20:28.912627 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:28.914063 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:28.973245 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c927d76-1455-4afe-97f3-70a8dee86f1a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:28.976853 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f90df-29c3-4b4c-9bf7-9ba5fc5b89f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:28.977521 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f90df-29c3-4b4c-9bf7-9ba5fc5b89f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:28.977806 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f90df-29c3-4b4c-9bf7-9ba5fc5b89f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:28.978599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f90df-29c3-4b4c-9bf7-9ba5fc5b89f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:28.987681 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f90df-29c3-4b4c-9bf7-9ba5fc5b89f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:28.989140 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f90df-29c3-4b4c-9bf7-9ba5fc5b89f9 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:28.999903 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1437/2873] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:29.294853 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc364-a214-4b9d-991c-f1ce6afdf30d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:29.296484 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ffdfc364-a214-4b9d-991c-f1ce6afdf30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:29.310570 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc364-a214-4b9d-991c-f1ce6afdf30d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:29.316425 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:29.317350 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:29.317634 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:29.320314 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] PATH_INFO: `/v3/projects/9da4240ffd66455cbdba47b9ac3b96bc` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:29.322854 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9da4240ffd66455cbdba47b9ac3b96bc)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:29.325604 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:29.378084 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9da4240ffd66455cbdba47b9ac3b96bc', 'request_id': 'req-a639b306-359d-4da4-8f30-134cbf7b6a34'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.384580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9da4240ffd66455cbdba47b9ac3b96bc', 'request_id': 'req-a639b306-359d-4da4-8f30-134cbf7b6a34'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.392836 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9da4240ffd66455cbdba47b9ac3b96bc', 'request_id': 'req-a639b306-359d-4da4-8f30-134cbf7b6a34'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.404460 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] The token cache is being invalidate because project 9da4240ffd66455cbdba47b9ac3b96bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:29.404753 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.405435 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.406990 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1437/2874] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:29 2022] DELETE /identity/v3/projects/9da4240ffd66455cbdba47b9ac3b96bc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:29.411161 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f90df-29c3-4b4c-9bf7-9ba5fc5b89f9 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:29.421740 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f90df-29c3-4b4c-9bf7-9ba5fc5b89f9 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:29.479333 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:29.480276 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:29.480598 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:29.480875 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] PATH_INFO: `/v3/users/a952a2d23fa742aea7173286ef2fd0fd` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:29.482326 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a952a2d23fa742aea7173286ef2fd0fd)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:29.484233 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:29.489784 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:29.532202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a952a2d23fa742aea7173286ef2fd0fd'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.544801 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a952a2d23fa742aea7173286ef2fd0fd'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.552423 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a952a2d23fa742aea7173286ef2fd0fd'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.553045 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a952a2d23fa742aea7173286ef2fd0fd'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.560386 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79c92b0b-f619-4acd-a03d-3ef89e521b1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a952a2d23fa742aea7173286ef2fd0fd'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:29.566407 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1438/2875] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:20:29 2022] DELETE /identity/v3/users/a952a2d23fa742aea7173286ef2fd0fd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:29.859390 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:29.861315 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:29.874749 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639b306-359d-4da4-8f30-134cbf7b6a34 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:29.951404 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:29.952322 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:29.952605 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:29.952884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] PATH_INFO: `/v3/projects/615599002a9b4c3ab760c3587014cfd7` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:29.954739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615599002a9b4c3ab760c3587014cfd7)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:29.956737 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:30.007489 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615599002a9b4c3ab760c3587014cfd7', 'request_id': 'req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:30.017275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615599002a9b4c3ab760c3587014cfd7', 'request_id': 'req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:30.027537 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615599002a9b4c3ab760c3587014cfd7', 'request_id': 'req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:30.034372 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:30.046879 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] The token cache is being invalidate because project 615599002a9b4c3ab760c3587014cfd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:20:30.047513 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:30.048728 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ba8c4563-c7f9-4e40-a218-5f3a5e0e5ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:20:30.051248 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1438/2876] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:20:29 2022] DELETE /identity/v3/projects/615599002a9b4c3ab760c3587014cfd7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:20:30.056802 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15c17b-ecdd-4d49-8c6c-1266e4ff47fa None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.057339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15c17b-ecdd-4d49-8c6c-1266e4ff47fa None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.057447 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15c17b-ecdd-4d49-8c6c-1266e4ff47fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.077533 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ac15c17b-ecdd-4d49-8c6c-1266e4ff47fa None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:30.078778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ac15c17b-ecdd-4d49-8c6c-1266e4ff47fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:30.171602 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1439/2877] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:30.179224 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c2316-c742-4dc5-90e7-4b34fa22a7b5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.179517 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c2316-c742-4dc5-90e7-4b34fa22a7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.179803 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c2316-c742-4dc5-90e7-4b34fa22a7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.211004 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-083c2316-c742-4dc5-90e7-4b34fa22a7b5 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:30.213316 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-083c2316-c742-4dc5-90e7-4b34fa22a7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:30.225150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1439/2878] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:30.227432 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15c17b-ecdd-4d49-8c6c-1266e4ff47fa None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:30.237473 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15c17b-ecdd-4d49-8c6c-1266e4ff47fa None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:30.242270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543df253-d5a0-4f88-bcfc-db8b182cf0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:30.242938 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543df253-d5a0-4f88-bcfc-db8b182cf0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.243222 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543df253-d5a0-4f88-bcfc-db8b182cf0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.244130 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543df253-d5a0-4f88-bcfc-db8b182cf0a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.245372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543df253-d5a0-4f88-bcfc-db8b182cf0a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:20:30.245372 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543df253-d5a0-4f88-bcfc-db8b182cf0a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:30.246716 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543df253-d5a0-4f88-bcfc-db8b182cf0a1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:30.260023 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1440/2879] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 19:20:30.265553 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c2316-c742-4dc5-90e7-4b34fa22a7b5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:30.275832 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c2316-c742-4dc5-90e7-4b34fa22a7b5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:30.280620 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a00cbb-8ba0-4dbc-a92e-20c5a415d9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:30.281326 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a00cbb-8ba0-4dbc-a92e-20c5a415d9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.281610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a00cbb-8ba0-4dbc-a92e-20c5a415d9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.281884 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a00cbb-8ba0-4dbc-a92e-20c5a415d9e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.282739 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a00cbb-8ba0-4dbc-a92e-20c5a415d9e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:30.284671 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a00cbb-8ba0-4dbc-a92e-20c5a415d9e9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:30.334153 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1440/2880] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:20:30 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:30.338778 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543df253-d5a0-4f88-bcfc-db8b182cf0a1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:30.356929 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543df253-d5a0-4f88-bcfc-db8b182cf0a1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:30.419289 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaabc6-b8a0-4b26-bd65-4d2a2a6eb8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:30.419993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaabc6-b8a0-4b26-bd65-4d2a2a6eb8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.420278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaabc6-b8a0-4b26-bd65-4d2a2a6eb8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.420557 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaabc6-b8a0-4b26-bd65-4d2a2a6eb8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.421288 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfaabc6-b8a0-4b26-bd65-4d2a2a6eb8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:30.423230 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfaabc6-b8a0-4b26-bd65-4d2a2a6eb8d3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:30.445489 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1441/2881] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:20:30 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:20:30.450585 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a00cbb-8ba0-4dbc-a92e-20c5a415d9e9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:30.461580 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a00cbb-8ba0-4dbc-a92e-20c5a415d9e9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:30.466427 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216ec6e-0a3d-48a2-b7c8-ab23b1f8d5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:30.467107 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216ec6e-0a3d-48a2-b7c8-ab23b1f8d5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.467401 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216ec6e-0a3d-48a2-b7c8-ab23b1f8d5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.467657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216ec6e-0a3d-48a2-b7c8-ab23b1f8d5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.468426 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216ec6e-0a3d-48a2-b7c8-ab23b1f8d5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:30.470325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216ec6e-0a3d-48a2-b7c8-ab23b1f8d5f0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:30.492573 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1441/2882] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:30.497370 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaabc6-b8a0-4b26-bd65-4d2a2a6eb8d3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:30.503174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:30.510283 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaabc6-b8a0-4b26-bd65-4d2a2a6eb8d3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:30.516239 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc723b4-78f3-4424-ac07-5003dccc119c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:30.516918 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc723b4-78f3-4424-ac07-5003dccc119c admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.517241 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc723b4-78f3-4424-ac07-5003dccc119c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.517527 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc723b4-78f3-4424-ac07-5003dccc119c admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.518258 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc723b4-78f3-4424-ac07-5003dccc119c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:30.519892 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc723b4-78f3-4424-ac07-5003dccc119c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:30.530775 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1442/2883] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:20:30.534984 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216ec6e-0a3d-48a2-b7c8-ab23b1f8d5f0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:30.546088 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216ec6e-0a3d-48a2-b7c8-ab23b1f8d5f0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:30.550915 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19ed48-8dc9-4b9d-bf45-dafadd92d61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:30.551814 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19ed48-8dc9-4b9d-bf45-dafadd92d61f admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.552098 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19ed48-8dc9-4b9d-bf45-dafadd92d61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.552377 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19ed48-8dc9-4b9d-bf45-dafadd92d61f admin admin] PATH_INFO: `/v3/projects/a212e3ffd5894cbbb3f6d8e05d0cc864/users/c025fdb326734b2d805e26f2233de432/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.570144 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19ed48-8dc9-4b9d-bf45-dafadd92d61f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a212e3ffd5894cbbb3f6d8e05d0cc864, user_id=c025fdb326734b2d805e26f2233de432, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:30.573275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19ed48-8dc9-4b9d-bf45-dafadd92d61f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:30.587988 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1442/2884] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:20:30 2022] PUT /identity/v3/projects/a212e3ffd5894cbbb3f6d8e05d0cc864/users/c025fdb326734b2d805e26f2233de432/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:20:30.600325 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c5db5-8c3f-416a-8a5b-f283dff147d8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.600628 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c5db5-8c3f-416a-8a5b-f283dff147d8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.600905 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c5db5-8c3f-416a-8a5b-f283dff147d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.634599 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d30c5db5-8c3f-416a-8a5b-f283dff147d8 None None] MFA Rules not processed for user `c025fdb326734b2d805e26f2233de432`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:20:30.636609 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d30c5db5-8c3f-416a-8a5b-f283dff147d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:30.707413 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1443/2885] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:20:30.719479 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19ed48-8dc9-4b9d-bf45-dafadd92d61f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:30.721368 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6c19ed48-8dc9-4b9d-bf45-dafadd92d61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:30.801825 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19ed48-8dc9-4b9d-bf45-dafadd92d61f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:30.805574 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e81e68-c77f-4a1c-bd2d-859a698b79c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915704f26c814eaa9e6c2a68533a9c66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:30.806791 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e81e68-c77f-4a1c-bd2d-859a698b79c7 service glance] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:30.807077 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e81e68-c77f-4a1c-bd2d-859a698b79c7 service glance] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:30.807356 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e81e68-c77f-4a1c-bd2d-859a698b79c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:30.816184 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e81e68-c77f-4a1c-bd2d-859a698b79c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:30.817684 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e81e68-c77f-4a1c-bd2d-859a698b79c7 service glance] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:30.827129 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1443/2886] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:20:40.704464 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=36 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:40.822858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:20:58.353429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c5db5-8c3f-416a-8a5b-f283dff147d8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:20:58.354848 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d30c5db5-8c3f-416a-8a5b-f283dff147d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:58.430009 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c5db5-8c3f-416a-8a5b-f283dff147d8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:20:58.434981 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c55ac-0ad7-4c02-8004-bf2112b2fda0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:20:58.435631 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c55ac-0ad7-4c02-8004-bf2112b2fda0 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:20:58.436595 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c55ac-0ad7-4c02-8004-bf2112b2fda0 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:20:58.436887 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c55ac-0ad7-4c02-8004-bf2112b2fda0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:20:58.438865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-308c55ac-0ad7-4c02-8004-bf2112b2fda0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:20:58.454568 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c55ac-0ad7-4c02-8004-bf2112b2fda0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:20:58.456215 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c55ac-0ad7-4c02-8004-bf2112b2fda0 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:20:58.468522 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1444/2887] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:20:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:21:03.959975 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e81e68-c77f-4a1c-bd2d-859a698b79c7 service glance] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:04.027828 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e81e68-c77f-4a1c-bd2d-859a698b79c7 service glance] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:04.032944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:04.033868 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:04.034151 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:04.034429 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] PATH_INFO: `/v3/users/c025fdb326734b2d805e26f2233de432` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:04.035679 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c025fdb326734b2d805e26f2233de432)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:04.037810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:04.086423 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c025fdb326734b2d805e26f2233de432'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.099104 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c025fdb326734b2d805e26f2233de432'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.108396 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c025fdb326734b2d805e26f2233de432'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.116170 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c025fdb326734b2d805e26f2233de432'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.116896 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c025fdb326734b2d805e26f2233de432'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.123194 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1444/2888] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:21:03 2022] DELETE /identity/v3/users/c025fdb326734b2d805e26f2233de432 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 19:21:04.128964 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895709-d58b-4336-9e5c-0718962a0dbb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:04.129785 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895709-d58b-4336-9e5c-0718962a0dbb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:04.129985 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895709-d58b-4336-9e5c-0718962a0dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:04.150441 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-57895709-d58b-4336-9e5c-0718962a0dbb None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:21:04.151712 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-57895709-d58b-4336-9e5c-0718962a0dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:04.214202 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1445/2889] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:21:04.225069 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:04.226603 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:04.288750 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17a3a1-ce82-43b8-bd7c-310387d37873 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:04.292694 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048d99f-252d-4dec-b972-b3e5ea2a0aa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:04.293380 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048d99f-252d-4dec-b972-b3e5ea2a0aa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:04.293669 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048d99f-252d-4dec-b972-b3e5ea2a0aa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:04.293950 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048d99f-252d-4dec-b972-b3e5ea2a0aa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:04.304125 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048d99f-252d-4dec-b972-b3e5ea2a0aa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:04.305622 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048d99f-252d-4dec-b972-b3e5ea2a0aa3 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:04.316593 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1445/2890] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:21:04.682025 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895709-d58b-4336-9e5c-0718962a0dbb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:04.683740 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-57895709-d58b-4336-9e5c-0718962a0dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:04.701155 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895709-d58b-4336-9e5c-0718962a0dbb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:04.708910 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83067350-5ade-4677-92be-77f48527125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:04.710359 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83067350-5ade-4677-92be-77f48527125c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:04.710703 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83067350-5ade-4677-92be-77f48527125c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:04.711070 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83067350-5ade-4677-92be-77f48527125c admin admin] PATH_INFO: `/v3/projects/a212e3ffd5894cbbb3f6d8e05d0cc864` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:04.712525 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83067350-5ade-4677-92be-77f48527125c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a212e3ffd5894cbbb3f6d8e05d0cc864)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:04.715216 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83067350-5ade-4677-92be-77f48527125c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:04.773047 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-83067350-5ade-4677-92be-77f48527125c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a212e3ffd5894cbbb3f6d8e05d0cc864', 'request_id': 'req-83067350-5ade-4677-92be-77f48527125c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.780470 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-83067350-5ade-4677-92be-77f48527125c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a212e3ffd5894cbbb3f6d8e05d0cc864', 'request_id': 'req-83067350-5ade-4677-92be-77f48527125c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.790327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-83067350-5ade-4677-92be-77f48527125c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a212e3ffd5894cbbb3f6d8e05d0cc864', 'request_id': 'req-83067350-5ade-4677-92be-77f48527125c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.805329 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-83067350-5ade-4677-92be-77f48527125c admin admin] The token cache is being invalidate because project a212e3ffd5894cbbb3f6d8e05d0cc864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:21:04.805625 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-83067350-5ade-4677-92be-77f48527125c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.806400 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-83067350-5ade-4677-92be-77f48527125c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:04.808344 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1446/2891] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:21:04 2022] DELETE /identity/v3/projects/a212e3ffd5894cbbb3f6d8e05d0cc864 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:21:04.813619 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32890bde-9432-4b76-8b0d-1b5e1f4f9de4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:04.813937 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32890bde-9432-4b76-8b0d-1b5e1f4f9de4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:04.814275 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32890bde-9432-4b76-8b0d-1b5e1f4f9de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:04.833962 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-32890bde-9432-4b76-8b0d-1b5e1f4f9de4 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:21:04.835220 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-32890bde-9432-4b76-8b0d-1b5e1f4f9de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:04.858285 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:21:04.907204 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1446/2892] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:21:04.915429 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ead09-1f20-42fe-8a5a-be1610d184e7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:04.915722 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ead09-1f20-42fe-8a5a-be1610d184e7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:04.915993 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ead09-1f20-42fe-8a5a-be1610d184e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:04.937660 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-421ead09-1f20-42fe-8a5a-be1610d184e7 None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:21:04.938894 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-421ead09-1f20-42fe-8a5a-be1610d184e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:04.947771 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1447/2893] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:21:04.952232 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32890bde-9432-4b76-8b0d-1b5e1f4f9de4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:04.962480 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32890bde-9432-4b76-8b0d-1b5e1f4f9de4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:04.967325 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb2979-ac92-423a-b4be-b30fb6cdb00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:04.967998 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb2979-ac92-423a-b4be-b30fb6cdb00e admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:04.968289 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb2979-ac92-423a-b4be-b30fb6cdb00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:04.968575 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb2979-ac92-423a-b4be-b30fb6cdb00e admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:04.969355 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb2979-ac92-423a-b4be-b30fb6cdb00e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 19:21:04.969691 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb2979-ac92-423a-b4be-b30fb6cdb00e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:04.971331 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb2979-ac92-423a-b4be-b30fb6cdb00e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:04.984009 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1447/2894] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jun 21 19:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 19:21:04.988914 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ead09-1f20-42fe-8a5a-be1610d184e7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:04.999270 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ead09-1f20-42fe-8a5a-be1610d184e7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:05.004169 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c81d6a-6dec-48db-9859-5fd39f536251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:05.004869 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c81d6a-6dec-48db-9859-5fd39f536251 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:05.005182 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c81d6a-6dec-48db-9859-5fd39f536251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:05.005466 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c81d6a-6dec-48db-9859-5fd39f536251 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:05.006174 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c81d6a-6dec-48db-9859-5fd39f536251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:05.008089 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c81d6a-6dec-48db-9859-5fd39f536251 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:05.011783 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:21:05.050662 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1448/2895] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jun 21 19:21:04 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 19:21:05.054894 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb2979-ac92-423a-b4be-b30fb6cdb00e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:05.066759 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb2979-ac92-423a-b4be-b30fb6cdb00e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:05.128944 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fcd0f-c1dd-4762-a9f3-25dbd6748011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:05.129657 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fcd0f-c1dd-4762-a9f3-25dbd6748011 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:05.129946 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fcd0f-c1dd-4762-a9f3-25dbd6748011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:05.130229 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fcd0f-c1dd-4762-a9f3-25dbd6748011 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:05.130965 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fcd0f-c1dd-4762-a9f3-25dbd6748011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:05.132921 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fcd0f-c1dd-4762-a9f3-25dbd6748011 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:05.154329 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1448/2896] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jun 21 19:21:05 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 19:21:05.159732 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c81d6a-6dec-48db-9859-5fd39f536251 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:05.170530 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c81d6a-6dec-48db-9859-5fd39f536251 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:05.175339 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9948073-0ea9-45dc-882a-d3e763e6082d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:05.175998 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9948073-0ea9-45dc-882a-d3e763e6082d admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:05.176285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9948073-0ea9-45dc-882a-d3e763e6082d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:05.176567 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9948073-0ea9-45dc-882a-d3e763e6082d admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:05.177327 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9948073-0ea9-45dc-882a-d3e763e6082d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:05.178937 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9948073-0ea9-45dc-882a-d3e763e6082d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:05.189958 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1449/2897] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:21:05.194870 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fcd0f-c1dd-4762-a9f3-25dbd6748011 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:05.207901 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fcd0f-c1dd-4762-a9f3-25dbd6748011 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:05.212587 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090a69c-9170-4dc0-9497-6dac77104db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:05.213277 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090a69c-9170-4dc0-9497-6dac77104db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:05.213566 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090a69c-9170-4dc0-9497-6dac77104db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:05.213848 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090a69c-9170-4dc0-9497-6dac77104db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:05.214569 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090a69c-9170-4dc0-9497-6dac77104db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:05.216164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090a69c-9170-4dc0-9497-6dac77104db8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:05.226160 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1449/2898] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jun 21 19:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 19:21:05.230789 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9948073-0ea9-45dc-882a-d3e763e6082d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:05.241003 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9948073-0ea9-45dc-882a-d3e763e6082d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:05.245816 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a59b9-2f5c-43c9-9463-b21d4fe481d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:05.246707 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a59b9-2f5c-43c9-9463-b21d4fe481d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:05.246997 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a59b9-2f5c-43c9-9463-b21d4fe481d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:05.247280 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a59b9-2f5c-43c9-9463-b21d4fe481d6 admin admin] PATH_INFO: `/v3/projects/e425387bfd65425c9050ec2f08f2121d/users/de80613dffb74bb29bc72b8080312685/roles/301682fc5c92440f874912eabfed6f70` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:05.266982 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a59b9-2f5c-43c9-9463-b21d4fe481d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e425387bfd65425c9050ec2f08f2121d, user_id=de80613dffb74bb29bc72b8080312685, role_id=301682fc5c92440f874912eabfed6f70)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:05.270035 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a59b9-2f5c-43c9-9463-b21d4fe481d6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:05.283787 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1450/2899] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jun 21 19:21:05 2022] PUT /identity/v3/projects/e425387bfd65425c9050ec2f08f2121d/users/de80613dffb74bb29bc72b8080312685/roles/301682fc5c92440f874912eabfed6f70 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:21:05.295218 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bba8fb-3d44-4fec-a19f-8ff0ed967857 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:05.295511 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bba8fb-3d44-4fec-a19f-8ff0ed967857 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:05.295797 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bba8fb-3d44-4fec-a19f-8ff0ed967857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:05.325610 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-45bba8fb-3d44-4fec-a19f-8ff0ed967857 None None] MFA Rules not processed for user `de80613dffb74bb29bc72b8080312685`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:21:05.328232 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-45bba8fb-3d44-4fec-a19f-8ff0ed967857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:05.403233 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1450/2900] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:21:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 19:21:05.416278 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a59b9-2f5c-43c9-9463-b21d4fe481d6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:05.418474 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-215a59b9-2f5c-43c9-9463-b21d4fe481d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:05.481253 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a59b9-2f5c-43c9-9463-b21d4fe481d6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:05.486577 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3beb411-406d-452c-836e-41ed6dba47b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8161446e1344108f8283d46da23ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:05.487285 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3beb411-406d-452c-836e-41ed6dba47b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:05.487579 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3beb411-406d-452c-836e-41ed6dba47b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:05.487855 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3beb411-406d-452c-836e-41ed6dba47b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:05.497845 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3beb411-406d-452c-836e-41ed6dba47b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:05.499323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3beb411-406d-452c-836e-41ed6dba47b4 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:05.508157 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1451/2901] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:21:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:21:15.400065 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:21:15.504208 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:21:25.968602 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bba8fb-3d44-4fec-a19f-8ff0ed967857 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:26.045817 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bba8fb-3d44-4fec-a19f-8ff0ed967857 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:26.050695 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:26.051588 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:26.051871 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:26.052150 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] PATH_INFO: `/v3/users/de80613dffb74bb29bc72b8080312685` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:26.053446 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de80613dffb74bb29bc72b8080312685)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:26.055409 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:26.112772 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de80613dffb74bb29bc72b8080312685'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.127164 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de80613dffb74bb29bc72b8080312685'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.137075 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de80613dffb74bb29bc72b8080312685'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.146810 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de80613dffb74bb29bc72b8080312685'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.147654 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de80613dffb74bb29bc72b8080312685'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.155478 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1451/2902] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jun 21 19:21:25 2022] DELETE /identity/v3/users/de80613dffb74bb29bc72b8080312685 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:21:26.160323 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc92d6-069c-47d3-928f-fe147513986c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:26.160876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc92d6-069c-47d3-928f-fe147513986c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:26.160876 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc92d6-069c-47d3-928f-fe147513986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:26.178978 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-97bc92d6-069c-47d3-928f-fe147513986c None None] MFA Rules not processed for user `8dcd62677ffc4535814a24f589fce966`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 19:21:26.180225 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-97bc92d6-069c-47d3-928f-fe147513986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:26.246287 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1452/2903] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jun 21 19:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 19:21:26.256816 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:26.258345 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:26.319724 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2960786-2b23-4763-9a43-c3ba5e14d5f8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:26.324060 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37674ed-1a86-41fa-b1c6-5914fa522a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '808b530d24b3487b9a20ee74c131502e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:26.325226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37674ed-1a86-41fa-b1c6-5914fa522a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:26.325226 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37674ed-1a86-41fa-b1c6-5914fa522a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:26.325477 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37674ed-1a86-41fa-b1c6-5914fa522a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:26.336071 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37674ed-1a86-41fa-b1c6-5914fa522a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:26.337583 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37674ed-1a86-41fa-b1c6-5914fa522a35 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:26.348708 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1452/2904] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:21:26.643173 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc92d6-069c-47d3-928f-fe147513986c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:21:26.645052 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-97bc92d6-069c-47d3-928f-fe147513986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:21:26.664176 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc92d6-069c-47d3-928f-fe147513986c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:21:26.671402 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dcd62677ffc4535814a24f589fce966', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8dc534a786d340398da9cb4c7a7e5355', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:21:26.672292 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:21:26.672580 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:21:26.672864 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] PATH_INFO: `/v3/projects/e425387bfd65425c9050ec2f08f2121d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:21:26.674163 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e425387bfd65425c9050ec2f08f2121d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:21:26.676136 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:21:26.734562 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e425387bfd65425c9050ec2f08f2121d', 'request_id': 'req-88dd4792-6f35-4d4a-81af-138ca3896781'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.741831 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e425387bfd65425c9050ec2f08f2121d', 'request_id': 'req-88dd4792-6f35-4d4a-81af-138ca3896781'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.751027 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e425387bfd65425c9050ec2f08f2121d', 'request_id': 'req-88dd4792-6f35-4d4a-81af-138ca3896781'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.763405 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] The token cache is being invalidate because project e425387bfd65425c9050ec2f08f2121d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 19:21:26.763751 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.764468 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-88dd4792-6f35-4d4a-81af-138ca3896781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 19:21:26.766207 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1453/2905] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jun 21 19:21:26 2022] DELETE /identity/v3/projects/e425387bfd65425c9050ec2f08f2121d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 19:21:36.343748 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:21:36.759865 dv-cinder-846303-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 19:26:01.899321 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37674ed-1a86-41fa-b1c6-5914fa522a35 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 19:26:01.901140 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e37674ed-1a86-41fa-b1c6-5914fa522a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:26:02.059466 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37674ed-1a86-41fa-b1c6-5914fa522a35 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 19:26:02.119959 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a7471-9ef1-4982-95d1-b955ec176e6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e3fc619dc74a0591d8ad14369ae673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0430e41e18bd4c2ebbca92c7816f8694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 19:26:02.120616 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a7471-9ef1-4982-95d1-b955ec176e6c service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 19:26:02.120914 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a7471-9ef1-4982-95d1-b955ec176e6c service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 19:26:02.121232 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a7471-9ef1-4982-95d1-b955ec176e6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 19:26:02.123186 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8a1a7471-9ef1-4982-95d1-b955ec176e6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 19:26:02.135154 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a7471-9ef1-4982-95d1-b955ec176e6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 19:26:02.136837 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a7471-9ef1-4982-95d1-b955ec176e6c service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 19:26:02.147858 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1453/2906] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jun 21 19:26:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 19:26:12.142711 dv-cinder-846303-1-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}